<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem: Sathya Narayan</title>
    <description>The latest articles on Forem by Sathya Narayan (@sathya_narayan).</description>
    <link>https://forem.com/sathya_narayan</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3051015%2F5ffbd3d3-e21f-47ab-9ef4-1e0f7ebb77ce.png</url>
      <title>Forem: Sathya Narayan</title>
      <link>https://forem.com/sathya_narayan</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed/sathya_narayan"/>
    <language>en</language>
    <item>
      <title>Connecting Azure Data Factory to a Mirrored Database in Microsoft Fabric (cross-tenant) Using a Service Principal</title>
      <dc:creator>Sathya Narayan</dc:creator>
      <pubDate>Mon, 14 Apr 2025 18:54:38 +0000</pubDate>
      <link>https://forem.com/sathya_narayan/connecting-azure-data-factory-to-a-mirrored-database-in-microsoft-fabric-cross-tenant-using-a-5akh</link>
      <guid>https://forem.com/sathya_narayan/connecting-azure-data-factory-to-a-mirrored-database-in-microsoft-fabric-cross-tenant-using-a-5akh</guid>
      <description>&lt;p&gt;&lt;strong&gt;Microsoft Fabric&lt;/strong&gt; allows data engineers to mirror databases such as Azure SQL or Cosmos DB directly into Fabric for real-time analytics. However, teams working with &lt;strong&gt;Azure Data Factory (ADF)&lt;/strong&gt; often need to use &lt;strong&gt;Service Principal (SP)&lt;/strong&gt; authentication to connect and extract data from these mirrored databases.&lt;/p&gt;

&lt;p&gt;In this post, I outline a working solution with all the steps, for cases where &lt;strong&gt;Azure Data Factory (ADF)&lt;/strong&gt; and &lt;strong&gt;Fabric&lt;/strong&gt; are in different tenants. The solution uses &lt;strong&gt;Service Principal (SP)&lt;/strong&gt; to authenticate across tenants.&lt;/p&gt;

&lt;p&gt;My current understanding, as of this writing (April 2025), is that &lt;strong&gt;ADF cannot directly connect to mirrored databases&lt;/strong&gt; in Fabric using the SQL endpoint and service principal authentication, even though:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The mirrored database exposes a SQL endpoint.&lt;/li&gt;
&lt;li&gt;The service principal has been granted access via Fabric.&lt;/li&gt;
&lt;li&gt;Connectivity is successful using tools like SSMS.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Solution Summary
&lt;/h2&gt;

&lt;p&gt;To enable ADF to access data from a mirrored database in Microsoft Fabric located in a different tenant:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Service principal authentication must be configured &lt;strong&gt;across tenants&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Since mirrored databases are not directly supported in ADF, &lt;strong&gt;the data must be copied into a native Fabric warehouse&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;ADF can then connect to the Fabric warehouse using the native connector with SP authentication.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Step-by-Step Implementation Guide
&lt;/h2&gt;

&lt;h3&gt;
  
  
  ① Configure the Service Principal
&lt;/h3&gt;

&lt;h4&gt;
  
  
  In the customer's (ADF-hosting) tenant:
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;Go to &lt;strong&gt;Azure Active Directory&lt;/strong&gt; &amp;gt; &lt;strong&gt;App registrations&lt;/strong&gt; &amp;gt; &lt;strong&gt;+ New registration&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Register an app and note:

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Application (client) ID&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Tenant ID&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Under &lt;strong&gt;Certificates &amp;amp; secrets&lt;/strong&gt;, generate a new &lt;strong&gt;Client Secret&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Under &lt;strong&gt;Authentication&lt;/strong&gt;, set the supported account types to:
&amp;gt; Accounts in any organizational directory (Multitenant)&lt;/li&gt;
&lt;/ol&gt;

&lt;h4&gt;
  
  
  In your tenant (Fabric-hosting):
&lt;/h4&gt;

&lt;ol&gt;
&lt;li&gt;Federate the service principal:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;az ad sp create &lt;span class="nt"&gt;--id&lt;/span&gt; &amp;lt;client-id&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;Retrieve the object ID:
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;az ad sp list &lt;span class="nt"&gt;--filter&lt;/span&gt; &lt;span class="s2"&gt;"appId eq '&amp;lt;client-id&amp;gt;'"&lt;/span&gt; &lt;span class="nt"&gt;--query&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt;0].id &lt;span class="nt"&gt;--output&lt;/span&gt; tsv
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h3&gt;
  
  
  ② Assign Fabric Access to the Service Principal
&lt;/h3&gt;

&lt;h4&gt;
  
  
  Using PowerShell:
&lt;/h4&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Install-Module&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;MicrosoftPowerBIMgmt&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Scope&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;CurrentUser&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Connect-PowerBIServiceAccount&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nx"&gt;Add-PowerBIWorkspaceUser&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Id&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;WorkspaceId&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="se"&gt;`
&lt;/span&gt;&lt;span class="w"&gt;                         &lt;/span&gt;&lt;span class="nt"&gt;-PrincipalType&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;App&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="se"&gt;`
&lt;/span&gt;&lt;span class="w"&gt;                         &lt;/span&gt;&lt;span class="nt"&gt;-Identifier&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;ObjectId&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="se"&gt;`
&lt;/span&gt;&lt;span class="w"&gt;                         &lt;/span&gt;&lt;span class="nt"&gt;-AccessRight&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Contributor&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;The workspace ID is the GUID of your Fabric workspace, visible in the URL.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Confirm access by checking the &lt;strong&gt;Manage Access&lt;/strong&gt; pane in Fabric.&lt;/p&gt;




&lt;h3&gt;
  
  
  ③ Copy Data from Mirrored Database to Fabric Warehouse
&lt;/h3&gt;

&lt;p&gt;Since ADF cannot read directly from a mirrored database, replicate its data into a &lt;strong&gt;Fabric warehouse&lt;/strong&gt;.&lt;/p&gt;

&lt;h4&gt;
  
  
  Option 1: Use T-SQL in Fabric
&lt;/h4&gt;

&lt;p&gt;Use the SQL endpoint of the Fabric mirrored DB to run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;warehouse&lt;/span&gt;&lt;span class="p"&gt;].[&lt;/span&gt;&lt;span class="k"&gt;table_name&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;mirrored&lt;/span&gt;&lt;span class="p"&gt;].[&lt;/span&gt;&lt;span class="n"&gt;source_table&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;Use &lt;code&gt;CREATE TABLE AS SELECT (CTAS)&lt;/code&gt; in your warehouse to replicate the table.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h4&gt;
  
  
  Option 2: Use Dataflows Gen2 or Pipelines inside Fabric
&lt;/h4&gt;

&lt;p&gt;Use Power Query or Fabric pipelines to stage data from the mirrored DB into the warehouse.&lt;/p&gt;




&lt;h3&gt;
  
  
  ④ Connect ADF to the Fabric Warehouse
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;In ADF, go to &lt;strong&gt;Manage&lt;/strong&gt; &amp;gt; &lt;strong&gt;Linked Services&lt;/strong&gt; &amp;gt; &lt;strong&gt;+ New&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Choose connector: &lt;strong&gt;Microsoft Fabric Warehouse&lt;/strong&gt; (or Azure SQL if Fabric connector is limited).&lt;/li&gt;
&lt;li&gt;Fill in:

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Server name&lt;/strong&gt;: SQL endpoint of the warehouse&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Database name&lt;/strong&gt;: Warehouse name&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Authentication&lt;/strong&gt;: Azure Active Directory - Service Principal&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tenant ID&lt;/strong&gt;: Your tenant (Fabric's tenant)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Client ID&lt;/strong&gt; + &lt;strong&gt;Client Secret&lt;/strong&gt;: From App Registration in the customer tenant&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Click &lt;strong&gt;Test Connection&lt;/strong&gt;.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Summary: What Works and What Doesn’t
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Capability&lt;/th&gt;
&lt;th&gt;Supported?&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;ADF connecting to Fabric Warehouse via SP&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Use native Fabric Warehouse connector&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ADF connecting to &lt;strong&gt;mirrored DB SQL endpoint&lt;/strong&gt; via SP&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;SQL endpoint exists but is not ADF-compatible&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSMS connecting to mirrored DB via SP&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Confirms permissions and network setup&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fabric SP assignment&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Must use PowerShell with Object ID&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OneLake/Delta export from mirrored DB&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Alternative if warehouse is not used&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;I hope this will help others who are trying to integrate cross-tenant SP-authenticated ADF pipelines into Fabric data platform.&lt;/p&gt;

</description>
      <category>learning</category>
      <category>azure</category>
      <category>microsoftfabric</category>
      <category>azuredatafactory</category>
    </item>
  </channel>
</rss>
