Browse Source

HADOOP-14548. S3Guard: documenting issues running parallel tests w/ s3n. Contributed by Aaron Fabbri.

Sean Mackrory 8 years ago
parent
commit
b8ca3bdf34

+ 8 - 2
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md

@@ -443,7 +443,7 @@ at a time until the operations appear to be working.
 mvn -T 1C verify -Dtest=skip -Dit.test=ITestS3AMiscOperations -Ds3guard -Ddynamo
 mvn -T 1C verify -Dtest=skip -Dit.test=ITestS3AMiscOperations -Ds3guard -Ddynamo
 ```
 ```
 
 
-Notes
+### Notes
 
 
 1. If the `s3guard` profile is not set, then the s3guard properties are those
 1. If the `s3guard` profile is not set, then the s3guard properties are those
 of the test configuration set in `contract-test-options.xml` or `auth-keys.xml`
 of the test configuration set in `contract-test-options.xml` or `auth-keys.xml`
@@ -452,8 +452,14 @@ If the `s3guard` profile *is* set,
 1. The s3guard options from maven (the dynamo and authoritative flags)
 1. The s3guard options from maven (the dynamo and authoritative flags)
   overwrite any previously set. in the configuration files.
   overwrite any previously set. in the configuration files.
 1. Dynamo will be configured to create any missing tables.
 1. Dynamo will be configured to create any missing tables.
-1. 
 
 
+### Warning About Concurrent Tests
+
+You should not run S3A and S3N tests in parallel on the same bucket.  This is
+especially true when S3Guard is enabled.  S3Guard requires that all clients
+that are modifying the bucket have S3Guard enabled, so having S3N
+integration tests running in parallel with S3A tests will cause strange
+failures.
 
 
 ### Scale Testing MetadataStore Directly
 ### Scale Testing MetadataStore Directly
 
 

+ 4 - 0
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md

@@ -107,6 +107,10 @@ each filesystem for its testing.
 1. `test.fs.s3n.name` : the URL of the bucket for S3n tests
 1. `test.fs.s3n.name` : the URL of the bucket for S3n tests
 1. `test.fs.s3a.name` : the URL of the bucket for S3a tests
 1. `test.fs.s3a.name` : the URL of the bucket for S3a tests
 
 
+*Note* that running s3a and s3n tests in parallel mode, against the same bucket
+is unreliable.  We recommend using separate buckets or testing one connector
+at a time.
+
 The contents of each bucket will be destroyed during the test process:
 The contents of each bucket will be destroyed during the test process:
 do not use the bucket for any purpose other than testing. Furthermore, for
 do not use the bucket for any purpose other than testing. Furthermore, for
 s3a, all in-progress multi-part uploads to the bucket will be aborted at the
 s3a, all in-progress multi-part uploads to the bucket will be aborted at the