Browse Source

HADOOP-14548. S3Guard: documenting issues running parallel tests w/ s3n. Contributed by Aaron Fabbri.

Sean Mackrory 8 years ago
parent
commit
b8ca3bdf34

+ 8 - 2
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/s3guard.md

@@ -443,7 +443,7 @@ at a time until the operations appear to be working.
 mvn -T 1C verify -Dtest=skip -Dit.test=ITestS3AMiscOperations -Ds3guard -Ddynamo
 ```
 
-Notes
+### Notes
 
 1. If the `s3guard` profile is not set, then the s3guard properties are those
 of the test configuration set in `contract-test-options.xml` or `auth-keys.xml`
@@ -452,8 +452,14 @@ If the `s3guard` profile *is* set,
 1. The s3guard options from maven (the dynamo and authoritative flags)
   overwrite any previously set. in the configuration files.
 1. Dynamo will be configured to create any missing tables.
-1. 
 
+### Warning About Concurrent Tests
+
+You should not run S3A and S3N tests in parallel on the same bucket.  This is
+especially true when S3Guard is enabled.  S3Guard requires that all clients
+that are modifying the bucket have S3Guard enabled, so having S3N
+integration tests running in parallel with S3A tests will cause strange
+failures.
 
 ### Scale Testing MetadataStore Directly
 

+ 4 - 0
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md

@@ -107,6 +107,10 @@ each filesystem for its testing.
 1. `test.fs.s3n.name` : the URL of the bucket for S3n tests
 1. `test.fs.s3a.name` : the URL of the bucket for S3a tests
 
+*Note* that running s3a and s3n tests in parallel mode, against the same bucket
+is unreliable.  We recommend using separate buckets or testing one connector
+at a time.
+
 The contents of each bucket will be destroyed during the test process:
 do not use the bucket for any purpose other than testing. Furthermore, for
 s3a, all in-progress multi-part uploads to the bucket will be aborted at the