|
|
@@ -14,8 +14,6 @@
|
|
|
|
|
|
# ABFS Driver for Namespace Disabled Accounts (FNS: Flat Namespace)
|
|
|
|
|
|
-### Note: FNS-BLOB Support is being built and not yet ready for usage.
|
|
|
-
|
|
|
## Background
|
|
|
The ABFS driver is recommended to be used only with HNS Enabled ADLS Gen-2 accounts
|
|
|
for big data analytics because of being more performant and scalable.
|
|
|
@@ -79,4 +77,80 @@ configured service type.
|
|
|
<name>fs.azure.fns.account.service.type</name>
|
|
|
<value>BLOB</value>
|
|
|
</property>
|
|
|
- ```
|
|
|
+ ```
|
|
|
+
|
|
|
+5. Auth type supported over FNS Blob: SharedKey, OAuth and fixed SAS.
|
|
|
+ ```xml
|
|
|
+ <property>
|
|
|
+ <name>fs.azure.account.auth.type</name>
|
|
|
+ <value>SharedKey</value>
|
|
|
+ </property>
|
|
|
+ ```
|
|
|
+ - How to configure Shared Key
|
|
|
+ auth: [Shared Key](./abfs.md#a-nameshared-key-autha-default-shared-key)
|
|
|
+ - How to configure
|
|
|
+ OAuth: [OAuth](./abfs.md#a-nameoauth-client-credentialsa-oauth-20-client-credentials)
|
|
|
+ - How to configure fixed
|
|
|
+ SAS: [Fixed SAS](./abfs.md#using-accountservice-sas-with-abfs)
|
|
|
+
|
|
|
+ OAuth is recommended auth type as it is more secure and flexible.
|
|
|
+
|
|
|
+### <a name="renamedeleteoptions"></a> Rename delete configs
|
|
|
+
|
|
|
+The following configs are related to rename and delete operations.
|
|
|
+
|
|
|
+- `fs.azure.blob.copy.progress.wait.millis`: Blob copy API is an async API, this
|
|
|
+ configuration defines polling duration for checking copy status. The default
|
|
|
+ value is 1 sec i.e. 1000 ms.
|
|
|
+
|
|
|
+- `fs.azure.blob.copy.max.wait.millis`: Maximum time to wait for a blob copy
|
|
|
+ operation to complete. The default value is 5 minutes.
|
|
|
+
|
|
|
+- `fs.azure.blob.atomic.rename.lease.refresh.duration`: Blob rename lease
|
|
|
+ refresh
|
|
|
+ duration in milliseconds. This setting ensures that the lease on the blob is
|
|
|
+ periodically refreshed during a rename operation to prevent other operations
|
|
|
+ from interfering.
|
|
|
+ The default value is 60 seconds.
|
|
|
+
|
|
|
+- `fs.azure.blob.dir.list.producer.queue.max.size`: Maximum number of blob
|
|
|
+ entries
|
|
|
+ enqueued in memory for rename or delete orchestration. The default value is 2
|
|
|
+ times the default value of list max results, which is 5000, making the current
|
|
|
+ value 10000.
|
|
|
+
|
|
|
+- `fs.azure.blob.dir.list.consumer.max.lag`: It sets a limit on how much blob
|
|
|
+ information can be waiting to be processed (consumer lag) during a blob
|
|
|
+ listing
|
|
|
+ operation. If the amount of unprocessed blob information exceeds this limit,
|
|
|
+ the
|
|
|
+ producer will pause until the consumer catches up and the lag becomes
|
|
|
+ manageable. The default value is equal to the value of default value of list
|
|
|
+ max
|
|
|
+ results which is 5000 currently.
|
|
|
+
|
|
|
+- `fs.azure.blob.dir.rename.max.thread`: Maximum number of threads per blob
|
|
|
+ rename
|
|
|
+ orchestration. The default value is 5.
|
|
|
+
|
|
|
+- `fs.azure.blob.dir.delete.max.thread`: Maximum number of thread per
|
|
|
+ blob-delete
|
|
|
+ orchestration. The default value currently is 5.
|
|
|
+
|
|
|
+## Features currently not supported
|
|
|
+
|
|
|
+1. **User Delegation SAS** feature is currently not supported but we
|
|
|
+ plan to bring support for it in the future.
|
|
|
+ Jira to track this
|
|
|
+ workitem : https://issues.apache.org/jira/browse/HADOOP-19406.
|
|
|
+
|
|
|
+
|
|
|
+2. **Context Provider Key (CPK)** support is currently not available. It refers to the ability to use a
|
|
|
+customer-provided encryption key to encrypt and decrypt data in Azure Blob
|
|
|
+Storage. This feature allows users to manage their own encryption keys,
|
|
|
+providing an additional layer of security and control over their data.
|
|
|
+
|
|
|
+## Ask all about ABFS Driver
|
|
|
+
|
|
|
+For any queries related to onboard to FNS Blob or anything related to ABFS
|
|
|
+Driver in general, kindly reach out to us at **askabfs@microsoft.com**.
|
Anmol Asrani