Преглед изворни кода

ZOOKEEPER-3956: Remove json-simple from ZooKeeper

JSON-Simple was only used in SnapshotFormatter in one place and also in Contrib's LogGraph component.
It's development has discontinued since 2012, so it's time to remove it.
I have replaced it with Jackson2, which we use in ZooKeeper anyway.
Added some unit tests to LogGraph as it didn't have any.

Author: Tamas Penzes <tamaas@cloudera.com>

Reviewers: Enrico Olivelli <eolivelli@apache.org>, Norbert Kalmar <nkalmar@apache.org>

Closes #1484 from tamaashu/ZOOKEEPER-3956
Tamas Penzes пре 4 година
родитељ
комит
83d79d16d6
24 измењених фајлова са 884 додато и 962 уклоњено
  1. 17 12
      pom.xml
  2. 0 4
      zookeeper-assembly/pom.xml
  3. 0 4
      zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml
  4. 28 18
      zookeeper-contrib/zookeeper-contrib-loggraph/README.md
  5. 0 70
      zookeeper-contrib/zookeeper-contrib-loggraph/build.xml
  6. 0 44
      zookeeper-contrib/zookeeper-contrib-loggraph/ivy.xml
  7. 46 2
      zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml
  8. 165 160
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java
  9. 125 155
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java
  10. 19 29
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java
  11. 42 38
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java
  12. 0 16
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java
  13. 53 44
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java
  14. 39 48
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java
  15. 85 88
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java
  16. 2 7
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh
  17. 2 7
      zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh
  18. 120 0
      zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FileLoaderTest.java
  19. 46 0
      zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FsTest.java
  20. 88 0
      zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/ThroughputTest.java
  21. 0 5
      zookeeper-server/pom.xml
  22. 7 5
      zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
  23. 0 4
      zookeeper-server/src/main/resources/LICENSE.txt
  24. 0 202
      zookeeper-server/src/main/resources/lib/json-simple-1.1.1.LICENSE.txt

+ 17 - 12
pom.xml

@@ -438,7 +438,6 @@
     <netty.version>4.1.50.Final</netty.version>
     <netty.version>4.1.50.Final</netty.version>
     <jetty.version>9.4.24.v20191120</jetty.version>
     <jetty.version>9.4.24.v20191120</jetty.version>
     <jackson.version>2.10.3</jackson.version>
     <jackson.version>2.10.3</jackson.version>
-    <json.version>1.1.1</json.version>
     <jline.version>2.14.6</jline.version>
     <jline.version>2.14.6</jline.version>
     <snappy.version>1.1.7</snappy.version>
     <snappy.version>1.1.7</snappy.version>
     <kerby.version>2.0.0</kerby.version>
     <kerby.version>2.0.0</kerby.version>
@@ -618,17 +617,6 @@
         <artifactId>jackson-databind</artifactId>
         <artifactId>jackson-databind</artifactId>
         <version>${jackson.version}</version>
         <version>${jackson.version}</version>
       </dependency>
       </dependency>
-      <dependency>
-        <groupId>com.googlecode.json-simple</groupId>
-        <artifactId>json-simple</artifactId>
-        <version>${json.version}</version>
-        <exclusions>
-          <exclusion>
-              <groupId>junit</groupId>
-              <artifactId>junit</artifactId>
-          </exclusion>
-        </exclusions>
-      </dependency>
       <dependency>
       <dependency>
         <groupId>jline</groupId>
         <groupId>jline</groupId>
         <artifactId>jline</artifactId>
         <artifactId>jline</artifactId>
@@ -1080,6 +1068,23 @@
               </rules>
               </rules>
             </configuration>
             </configuration>
           </execution>
           </execution>
+          <execution>
+            <id>banned-json-simple</id>
+            <goals>
+              <goal>enforce</goal>
+            </goals>
+            <configuration>
+              <rules>
+                <bannedDependencies>
+                  <excludes>
+                    <exclude>com.googlecode.json-simple:json-simple</exclude>
+                  </excludes>
+                  <searchTransitive>false</searchTransitive>
+                  <message>We don't use json-simple any more, so do not depend on it directly.</message>
+                </bannedDependencies>
+              </rules>
+            </configuration>
+          </execution>
         </executions>
         </executions>
       </plugin>
       </plugin>
     </plugins>
     </plugins>

+ 0 - 4
zookeeper-assembly/pom.xml

@@ -99,10 +99,6 @@
       <groupId>com.fasterxml.jackson.core</groupId>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <artifactId>jackson-databind</artifactId>
     </dependency>
     </dependency>
-    <dependency>
-      <groupId>com.googlecode.json-simple</groupId>
-      <artifactId>json-simple</artifactId>
-    </dependency>
     <dependency>
     <dependency>
       <groupId>jline</groupId>
       <groupId>jline</groupId>
       <artifactId>jline</artifactId>
       <artifactId>jline</artifactId>

+ 0 - 4
zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml

@@ -78,10 +78,6 @@
       <groupId>com.fasterxml.jackson.core</groupId>
       <groupId>com.fasterxml.jackson.core</groupId>
       <artifactId>jackson-databind</artifactId>
       <artifactId>jackson-databind</artifactId>
     </dependency>
     </dependency>
-    <dependency>
-      <groupId>com.googlecode.json-simple</groupId>
-      <artifactId>json-simple</artifactId>
-    </dependency>
     <dependency>
     <dependency>
       <groupId>jline</groupId>
       <groupId>jline</groupId>
       <artifactId>jline</artifactId>
       <artifactId>jline</artifactId>

+ 28 - 18
zookeeper-contrib/zookeeper-contrib-loggraph/README.txt → zookeeper-contrib/zookeeper-contrib-loggraph/README.md

@@ -1,25 +1,27 @@
-LogGraph README
+# LogGraph README
 
 
-1 - About
+## 1 - About
 LogGraph is an application for viewing and filtering zookeeper logs. It can handle transaction logs and message logs. 
 LogGraph is an application for viewing and filtering zookeeper logs. It can handle transaction logs and message logs. 
 
 
-2 - Compiling
+## 2 - Compiling
 
 
-Run "ant jar" in src/contrib/loggraph/. This will download all dependencies and compile all the loggraph code.
+Run `mvn clean install dependency:copy-dependencies` in zookeeper-contrib/zookeeper-contrib-loggraph/.
+This will download all dependencies and compile all the loggraph code.
 
 
-Once compilation has finished, you can run it the the loggraph.sh script in zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources.
+Once compilation has finished, you can run it the the `loggraph.sh` script in zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources.
 This will start and embedded web server on your machine.
 This will start and embedded web server on your machine.
-Navigate to http://localhost:8182/graph/main.html
+Navigate to `http://localhost:8182/graph/main.html`
 
 
-3 - Usage
+## 3 - Usage
 LogGraph presents the user with 4 views, 
 LogGraph presents the user with 4 views, 
  
  
   a) Simple log view
   a) Simple log view
      This view simply displays the log text. This isn't very useful without filters (see "Filtering the logs").
      This view simply displays the log text. This isn't very useful without filters (see "Filtering the logs").
 
 
   b) Server view
   b) Server view
-     The server view shows the interactions between the different servers in an ensemble. The X axis represents time. 
-        * Exceptions show up as red dots. Hovering your mouse over them will give you more details of the exception
+     The server view shows the interactions between the different servers in an ensemble. The X axis represents time.
+      
+    * Exceptions show up as red dots. Hovering your mouse over them will give you more details of the exception
 	* The colour of the line represents the election state of the server. 
 	* The colour of the line represents the election state of the server. 
 	   - orange means LOOKING for leader
 	   - orange means LOOKING for leader
 	   - dark green means the server is the leader
 	   - dark green means the server is the leader
@@ -34,37 +36,45 @@ LogGraph presents the user with 4 views,
   d) Stats view
   d) Stats view
      There is currently only one statistics view, Transactions/minute. Suggestions for other statistic views are very welcome.
      There is currently only one statistics view, Transactions/minute. Suggestions for other statistic views are very welcome.
 
 
-4 - Filtering the logs
+## 4 - Filtering the logs
 The logs can be filtered in 2 ways, by time and by content. 
 The logs can be filtered in 2 ways, by time and by content. 
 
 
 To filter by time simply move the slider to the desired start time. The time window specifies how many milliseconds after and including the start time will be displayed.
 To filter by time simply move the slider to the desired start time. The time window specifies how many milliseconds after and including the start time will be displayed.
 
 
 Content filtering uses a adhoc filtering language, using prefix notation. The language looks somewhat similar to lisp. A statement in the language takes the form (op arg arg ....). A statement resolves to a boolean value. Statements can be nested. 
 Content filtering uses a adhoc filtering language, using prefix notation. The language looks somewhat similar to lisp. A statement in the language takes the form (op arg arg ....). A statement resolves to a boolean value. Statements can be nested. 
 
 
-4.1 - Filter arguments
+### 4.1 - Filter arguments
 An argument can be a number, a string or a symbol. A number is any argument which starts with -, + or 0 to 9. If the number starts with 0x it is interpretted as hexidecimal. Otherwise it is interpretted as decimal. If the argument begins with a double-quote, (") it is interpretted as a string. Anything else is interpretted as a symbol.
 An argument can be a number, a string or a symbol. A number is any argument which starts with -, + or 0 to 9. If the number starts with 0x it is interpretted as hexidecimal. Otherwise it is interpretted as decimal. If the argument begins with a double-quote, (") it is interpretted as a string. Anything else is interpretted as a symbol.
 
 
-4.2 - Filter symbols
+### 4.2 - Filter symbols
 The possible filter symbols are: 
 The possible filter symbols are: 
 
 
 client-id : number, the session id of the client who initiated a transaction.
 client-id : number, the session id of the client who initiated a transaction.
+
 cxid : number, the cxid of a transaction
 cxid : number, the cxid of a transaction
+
 zxid : number, the zxid of a transaction
 zxid : number, the zxid of a transaction
+
 operation : string, the operation being performed, for example "setData", "createSession", "closeSession", "error", "create"
 operation : string, the operation being performed, for example "setData", "createSession", "closeSession", "error", "create"
 
 
-4.3 - Filter operations
+### 4.3 - Filter operations
 The possible filter operations are:
 The possible filter operations are:
 
 
 or : logical or, takes 1 or more arguments which must be other statements.
 or : logical or, takes 1 or more arguments which must be other statements.
+
 and : logical and, takes 1 or more arguments which must be other statements.
 and : logical and, takes 1 or more arguments which must be other statements.
+
 not : logical not, takes 1 argument which must be another statement.
 not : logical not, takes 1 argument which must be another statement.
+
 xor : exclusive or, takes 1 or more arguments which must be other statements.
 xor : exclusive or, takes 1 or more arguments which must be other statements.
-= : equals, takes 1 or more arguments, which must all be equal to each other to return true.
-> : greater than, takes 1 or more arguments, to return true the 1st argument must be greater than the 2nd argument which must be greater than the 3rd argument and so on... 
-< : less than, takes 1 or more arguments, to return true the 1st argument must be less than the 2nd argument which must be less than the 3rd argument and so on... 
 
 
-4.3 - Filter examples
+ = : equals, takes 1 or more arguments, which must all be equal to each other to return true.
+
+&gt; : greater than, takes 1 or more arguments, to return true the 1st argument must be greater than the 2nd argument which must be greater than the 3rd argument and so on... 
+
+&lt; : less than, takes 1 or more arguments, to return true the 1st argument must be less than the 2nd argument which must be less than the 3rd argument and so on... 
+
+### 4.4 - Filter examples
 Give me all the setData operations with session id 0xdeadbeef or 0xcafeb33r but not with zxid 0x12341234 ->
 Give me all the setData operations with session id 0xdeadbeef or 0xcafeb33r but not with zxid 0x12341234 ->
 
 
 (and (= operation "setData") (or (= client-id 0xdeadbeef) (= client-id 0xcafeb33r)) (not (= zxid 0x12341234)))
 (and (= operation "setData") (or (= client-id 0xdeadbeef) (= client-id 0xcafeb33r)) (not (= zxid 0x12341234)))
-

+ 0 - 70
zookeeper-contrib/zookeeper-contrib-loggraph/build.xml

@@ -1,70 +0,0 @@
-<?xml version="1.0"?>
-
-<!--
-   Licensed to the Apache Software Foundation (ASF) under one or more
-   contributor license agreements.  See the NOTICE file distributed with
-   this work for additional information regarding copyright ownership.
-   The ASF licenses this file to You under the Apache License, Version 2.0
-   (the "License"); you may not use this file except in compliance with
-   the License.  You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
--->
-
-<project name="loggraph" default="jar">
-
-  <import file="../build-contrib.xml"/>
-  
-  <target name="init" depends="check-contrib,zookeeperbuildcontrib.init" unless="skip.contrib">
-    <echo message="contrib: ${name}"/>
-    <mkdir dir="${build.dir}"/>
-    <antcall target="init-contrib"/>
-  </target>
-
-  <target name="compile" depends="init,ivy-retrieve,zookeeperbuildcontrib.compile" unless="skip.contrib">
-  </target>
-
-  <target name="setjarname">
-    <property name="jarname" value="${build.dir}/zookeeper-${version}-${name}.jar"/>
-  </target>
-
-  <target name="jar" depends="setjarname,compile"  >
-    <jar destfile="${jarname}">
-      <fileset file="${zk.root}/LICENSE.txt" />
-      <fileset dir="${build.classes}" />
-      <fileset dir="src/main/resources/webapp"/>
-      <manifest>
-        <attribute name="Built-By" value="${user.name}"/>
-        <attribute name="Built-At" value="${build.time}"/>
-        <attribute name="Built-On" value="${host.name}" />
-        <attribute name="Implementation-Title" value="org.apache.zookeeper.graph"/>
-        <attribute name="Implementation-Version" value="${revision}"/>
-        <attribute name="Implementation-Vendor" value="The Apache Software Foundation"/>
-      </manifest>
-    </jar>
-  </target>
-  
-  <target name="test">
-    <echo message="No test target defined for this package" />
-  </target>
-  
-
-  <target name="package" depends="compile, zookeeperbuildcontrib.package" unless="skip.contrib">
-    <echo message="contrib: ${name}"/>
-    
-    <copy file="${basedir}/build.xml" todir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}"/>
-
-    <mkdir dir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}/src"/>
-    <copy todir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}/src">
-      <fileset dir="${basedir}/src/main"/>
-    </copy>
-
-  </target>
-
-</project>

+ 0 - 44
zookeeper-contrib/zookeeper-contrib-loggraph/ivy.xml

@@ -1,44 +0,0 @@
-<!--
-   Licensed to the Apache Software Foundation (ASF) under one or more
-   contributor license agreements.  See the NOTICE file distributed with
-   this work for additional information regarding copyright ownership.
-   The ASF licenses this file to You under the Apache License, Version 2.0
-   (the "License"); you may not use this file except in compliance with
-   the License.  You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
--->
-
-<ivy-module version="2.0"
-            xmlns:e="http://ant.apache.org/ivy/extra">
-
-  <info organisation="org.apache.zookeeper"
-        module="${name}" revision="${version}">
-    <license name="Apache 2.0"/>
-    <ivyauthor name="Apache ZooKeeper" url="http://zookeeper.apache.org"/>
-    <description>ZooKeeper Graphing</description>
-  </info>
-
-  <configurations defaultconfmapping="default">
-    <conf name="default"/>
-    <conf name="test"/>
-  </configurations>
-
-  <dependencies>
-    <dependency org="org.slf4j" name="slf4j-api" rev="1.7.5"/>
-    <dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" transitive="false"/>
-  
-    <!-- transitive false turns off dependency checking, log4j deps seem borked -->
-    <dependency org="log4j" name="log4j" rev="1.2.17" transitive="false"/>
-    <dependency org="org.eclipse.jetty" name="jetty-server" rev="9.2.18.v20160721" />
-    <dependency org="org.eclipse.jetty" name="jetty-servlet" rev="9.2.18.v20160721" />
-    <dependency org="com.googlecode.json-simple" name="json-simple" rev="1.1" />
-  </dependencies>
-
-</ivy-module>

+ 46 - 2
zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml

@@ -44,6 +44,10 @@
       <artifactId>zookeeper</artifactId>
       <artifactId>zookeeper</artifactId>
       <version>${project.version}</version>
       <version>${project.version}</version>
     </dependency>
     </dependency>
+    <dependency>
+      <groupId>com.fasterxml.jackson.core</groupId>
+      <artifactId>jackson-databind</artifactId>
+    </dependency>
     <dependency>
     <dependency>
       <groupId>org.slf4j</groupId>
       <groupId>org.slf4j</groupId>
       <artifactId>slf4j-api</artifactId>
       <artifactId>slf4j-api</artifactId>
@@ -77,8 +81,19 @@
       <artifactId>jetty-servlet</artifactId>
       <artifactId>jetty-servlet</artifactId>
     </dependency>
     </dependency>
     <dependency>
     <dependency>
-      <groupId>com.googlecode.json-simple</groupId>
-      <artifactId>json-simple</artifactId>
+      <groupId>org.junit.jupiter</groupId>
+      <artifactId>junit-jupiter-api</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.junit.jupiter</groupId>
+      <artifactId>junit-jupiter-engine</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mockito</groupId>
+      <artifactId>mockito-core</artifactId>
+      <scope>test</scope>
     </dependency>
     </dependency>
   </dependencies>
   </dependencies>
 
 
@@ -88,6 +103,35 @@
         <directory>${project.basedir}/src/main/resources/webapp</directory>
         <directory>${project.basedir}/src/main/resources/webapp</directory>
       </resource>
       </resource>
     </resources>
     </resources>
+    <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-dependency-plugin</artifactId>
+        <configuration>
+          <outputDirectory>
+            ${project.basedir}/lib
+          </outputDirectory>
+        </configuration>
+      </plugin>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-surefire-plugin</artifactId>
+        <configuration>
+          <includes>
+            <include>**/*Test.java</include>
+          </includes>
+          <forkCount>${surefire-forkcount}</forkCount>
+          <reuseForks>false</reuseForks>
+          <argLine>-Xmx512m -Dtest.junit.threads=${surefire-forkcount} -Dzookeeper.junit.threadid=${surefire.forkNumber}</argLine>
+          <basedir>${project.basedir}</basedir>
+          <redirectTestOutputToFile>true</redirectTestOutputToFile>
+          <systemPropertyVariables>
+            <build.test.dir>${project.build.directory}/surefire</build.test.dir>
+            <zookeeper.DigestAuthenticationProvider.superDigest>super:D/InIHSb7yEEbrWz8b9l71RjZJU=</zookeeper.DigestAuthenticationProvider.superDigest>
+          </systemPropertyVariables>
+        </configuration>
+      </plugin>
+    </plugins>
   </build>
   </build>
 
 
 </project>
 </project>

+ 165 - 160
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java

@@ -17,25 +17,22 @@
  */
  */
 package org.apache.zookeeper.graph;
 package org.apache.zookeeper.graph;
 
 
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
-import java.io.Writer;
-import java.io.OutputStreamWriter;
-import java.io.IOException;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 import java.util.regex.Pattern;
 import java.util.regex.Pattern;
 import java.util.regex.Matcher;
 import java.util.regex.Matcher;
 import java.util.HashSet;
 import java.util.HashSet;
 import java.util.Iterator;
 import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.ListIterator;
 import java.util.Set;
 import java.util.Set;
 
 
 public class JsonGenerator {
 public class JsonGenerator {
-    private JSONObject root;
-    private Set<Integer> servers;
+	private final ObjectMapper mapper = new ObjectMapper();
+	private final JsonNode root;
+    private final Set<Integer> servers;
 
 
     private class Message {
     private class Message {
 	private int from;
 	private int from;
@@ -55,170 +52,178 @@ public class JsonGenerator {
 	}
 	}
     };
     };
 
 
-    public JSONObject txnEntry(TransactionEntry e) {
-	JSONObject event = new JSONObject();
+    public JsonNode txnEntry(TransactionEntry e) {
+		JsonNode event = mapper.createObjectNode();
 
 
-	event.put("time", Long.toString(e.getTimestamp()));
-	event.put("client", Long.toHexString(e.getClientId()));
-	event.put("cxid", Long.toHexString(e.getCxid()));
-	event.put("zxid", Long.toHexString(e.getZxid()));
-	event.put("op", e.getOp());
-	event.put("extra", e.getExtra());
-	event.put("type", "transaction");
+		((ObjectNode) event).put("time", Long.toString(e.getTimestamp()));
+		((ObjectNode) event).put("client", Long.toHexString(e.getClientId()));
+		((ObjectNode) event).put("cxid", Long.toHexString(e.getCxid()));
+		((ObjectNode) event).put("zxid", Long.toHexString(e.getZxid()));
+		((ObjectNode) event).put("op", e.getOp());
+		((ObjectNode) event).put("extra", e.getExtra());
+		((ObjectNode) event).put("type", "transaction");
 
 
-	return event;
+		return event;
     }
     }
 
 
     /**
     /**
        Assumes entries are sorted by timestamp.
        Assumes entries are sorted by timestamp.
      */
      */
     public JsonGenerator(LogIterator iter) {
     public JsonGenerator(LogIterator iter) {
-	servers = new HashSet<Integer>();
+		servers = new HashSet<Integer>();
 
 
-	Pattern stateChangeP = Pattern.compile("- (LOOKING|FOLLOWING|LEADING)");
-	Pattern newElectionP = Pattern.compile("New election. My id =  (\\d+), Proposed zxid = (\\d+)");
-	Pattern receivedProposalP = Pattern.compile("Notification: (\\d+) \\(n.leader\\), (\\d+) \\(n.zxid\\), (\\d+) \\(n.round\\), .+ \\(n.state\\), (\\d+) \\(n.sid\\), .+ \\(my state\\)");
-	Pattern exceptionP = Pattern.compile("xception");
-	
-	root = new JSONObject();
-	Matcher m = null;
-	JSONArray events = new JSONArray();
-	root.put("events", events);
-	
-	long starttime = Long.MAX_VALUE;
-	long endtime = 0;
-
-	int leader = 0;
-	long curEpoch = 0;
-	boolean newEpoch = false;
-
-	while (iter.hasNext()) {
-	    LogEntry ent = iter.next();
-	    
-	    if (ent.getTimestamp() < starttime) {
-		starttime = ent.getTimestamp();
-	    }
-	    if (ent.getTimestamp() > endtime) {
-		endtime = ent.getTimestamp();
-	    }
-	    
-	    if (ent.getType() == LogEntry.Type.TXN) {
-		events.add(txnEntry((TransactionEntry)ent));
-	    } else {
-		Log4JEntry e = (Log4JEntry)ent;
-		servers.add(e.getNode());
-		
-		if ((m = stateChangeP.matcher(e.getEntry())).find()) {
-		    JSONObject stateChange = new JSONObject();
-		    stateChange.put("type", "stateChange");
-		    stateChange.put("time", e.getTimestamp());
-		    stateChange.put("server", e.getNode());
-		    stateChange.put("state", m.group(1));
-		    events.add(stateChange);
-		    
-		    if (m.group(1).equals("LEADING")) {
-			leader = e.getNode();
-		    }
-		} else if ((m = newElectionP.matcher(e.getEntry())).find()) {
-		    Iterator<Integer> iterator = servers.iterator();
-		    long zxid = Long.valueOf(m.group(2));
-		    int count = (int)zxid;// & 0xFFFFFFFFL;
-		    int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
-		    
-		    if (leader != 0 && epoch > curEpoch) {
-			JSONObject stateChange = new JSONObject();
-			stateChange.put("type", "stateChange");
-			stateChange.put("time", e.getTimestamp());
-			stateChange.put("server", leader);
-			stateChange.put("state", "INIT");
-			events.add(stateChange);
-			leader = 0;
-		    }
-		    
-		    while (iterator.hasNext()) {
-			int dst = iterator.next();
-			if (dst != e.getNode()) {
-			    JSONObject msg = new JSONObject();
-			    msg.put("type", "postmessage");
-			    msg.put("src", e.getNode());
-			    msg.put("dst", dst);
-			    msg.put("time", e.getTimestamp());
-			    msg.put("zxid", m.group(2));
-			    msg.put("count", count);
-			    msg.put("epoch", epoch);
-			    
-			    events.add(msg);
+		Pattern stateChangeP = Pattern.compile("- (LOOKING|FOLLOWING|LEADING)");
+		Pattern newElectionP = Pattern.compile("New election. My id =  (\\d+), Proposed zxid = (\\d+)");
+		Pattern receivedProposalP = Pattern.compile("Notification: (\\d+) \\(n.leader\\), (\\d+) \\(n.zxid\\), (\\d+) \\(n.round\\), .+ \\(n.state\\), (\\d+) \\(n.sid\\), .+ \\(my state\\)");
+		Pattern exceptionP = Pattern.compile("xception");
+
+		root = mapper.createObjectNode();
+		Matcher m = null;
+		ArrayNode events = mapper.createArrayNode();
+		((ObjectNode)root).set("events", events);
+
+		long starttime = Long.MAX_VALUE;
+		long endtime = 0;
+
+		int leader = 0;
+		long curEpoch = 0;
+
+		while (iter.hasNext()) {
+			LogEntry ent = iter.next();
+
+			if (ent.getTimestamp() < starttime) {
+				starttime = ent.getTimestamp();
+			}
+			if (ent.getTimestamp() > endtime) {
+				endtime = ent.getTimestamp();
 			}
 			}
-		    }
-		} else if ((m = receivedProposalP.matcher(e.getEntry())).find()) {
-		    // Pattern.compile("Notification: \\d+, (\\d+), (\\d+), \\d+, [^,]*, [^,]*, (\\d+)");//, LOOKING, LOOKING, 2
-		    int src = Integer.valueOf(m.group(4));
-		    long zxid = Long.valueOf(m.group(2));
-		    int dst = e.getNode();
-		    long epoch2 = Long.valueOf(m.group(3));
-		    
-		    int count = (int)zxid;// & 0xFFFFFFFFL;
-		    int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
-		    
-		    if (leader != 0 && epoch > curEpoch) {
-			JSONObject stateChange = new JSONObject();
-			stateChange.put("type", "stateChange");
-			stateChange.put("time", e.getTimestamp());
-			stateChange.put("server", leader);
-			stateChange.put("state", "INIT");
-			events.add(stateChange);
-			leader = 0;
-		    }
-		    
-		    if (src != dst) {
-			JSONObject msg = new JSONObject();
-			msg.put("type", "delivermessage");
-			msg.put("src", src);
-			msg.put("dst", dst);
-			msg.put("time", e.getTimestamp());
-			msg.put("zxid", zxid);
-			msg.put("epoch", epoch);
-			msg.put("count", count);
-			msg.put("epoch2", epoch2);
-			
-			events.add(msg);
-		    }
-		} else if ((m = exceptionP.matcher(e.getEntry())).find()) {
-		    JSONObject ex = new JSONObject();
-		    ex.put("type", "exception");
-		    ex.put("server", e.getNode());
-		    ex.put("time", e.getTimestamp());
-		    ex.put("text", e.getEntry());
-		    events.add(ex);
-		} 
-	    }
-	    JSONObject ex = new JSONObject();
-	    ex.put("type", "text");
-	    ex.put("time", ent.getTimestamp());
-	    String txt = ent.toString();
-	    ex.put("text", txt);
-	    events.add(ex);
-	}
-	//	System.out.println("pending messages: "+pendingMessages.size());
-	root.put("starttime", starttime);
-	root.put("endtime", endtime);
 
 
-	JSONArray serversarray = new JSONArray();
-	root.put("servers", serversarray);
-	
-	Iterator<Integer> iterator = servers.iterator();
-	while (iterator.hasNext()) {
-	    serversarray.add(iterator.next());
-	}
+			if (ent.getType() == LogEntry.Type.TXN) {
+				events.add(txnEntry((TransactionEntry)ent));
+			}
+			else {
+				Log4JEntry e = (Log4JEntry)ent;
+				servers.add(e.getNode());
+
+				if ((m = stateChangeP.matcher(e.getEntry())).find()) {
+					JsonNode stateChange = add("stateChange", e.getTimestamp(), e.getNode(), m.group(1));
+					events.add(stateChange);
+
+					if (m.group(1).equals("LEADING")) {
+					leader = e.getNode();
+					}
+				}
+				else if ((m = newElectionP.matcher(e.getEntry())).find()) {
+					Iterator<Integer> iterator = servers.iterator();
+					long zxid = Long.valueOf(m.group(2));
+					int count = (int)zxid;// & 0xFFFFFFFFL;
+					int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+
+					if (leader != 0 && epoch > curEpoch) {
+						JsonNode stateChange = add("stateChange", e.getTimestamp(), leader, "INIT");
+						events.add(stateChange);
+
+						leader = 0;
+					}
+
+					while (iterator.hasNext()) {
+						int dst = iterator.next();
+						if (dst != e.getNode()) {
+							JsonNode msg = mapper.createObjectNode();
+							((ObjectNode)msg).put("type", "postmessage");
+							((ObjectNode)msg).put("src", e.getNode());
+							((ObjectNode)msg).put("dst", dst);
+							((ObjectNode)msg).put("time", e.getTimestamp());
+							((ObjectNode)msg).put("zxid", m.group(2));
+							((ObjectNode)msg).put("count", count);
+							((ObjectNode)msg).put("epoch", epoch);
+
+							events.add(msg);
+						}
+					}
+				}
+				else if ((m = receivedProposalP.matcher(e.getEntry())).find()) {
+					// Pattern.compile("Notification: \\d+, (\\d+), (\\d+), \\d+, [^,]*, [^,]*, (\\d+)");//, LOOKING, LOOKING, 2
+					int src = Integer.valueOf(m.group(4));
+					long zxid = Long.valueOf(m.group(2));
+					int dst = e.getNode();
+					long epoch2 = Long.valueOf(m.group(3));
+
+					int count = (int)zxid;// & 0xFFFFFFFFL;
+					int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+
+					if (leader != 0 && epoch > curEpoch) {
+						JsonNode stateChange = add("stateChange", e.getTimestamp(), leader, "INIT");
+						events.add(stateChange);
+
+						leader = 0;
+					}
+
+					if (src != dst) {
+						JsonNode msg = mapper.createObjectNode();
+						((ObjectNode)msg).put("type", "delivermessage");
+						((ObjectNode)msg).put("src", src);
+						((ObjectNode)msg).put("dst", dst);
+						((ObjectNode)msg).put("time", e.getTimestamp());
+						((ObjectNode)msg).put("zxid", zxid);
+						((ObjectNode)msg).put("count", count);
+						((ObjectNode)msg).put("epoch", epoch);
+						((ObjectNode)msg).put("epoch2", epoch2);
+
+					events.add(msg);
+					}
+				}
+				else if ((m = exceptionP.matcher(e.getEntry())).find()) {
+					JsonNode ex = mapper.createObjectNode();
+					((ObjectNode)ex).put("type", "exception");
+					((ObjectNode)ex).put("time", e.getTimestamp());
+					((ObjectNode)ex).put("server", e.getNode());
+					((ObjectNode)ex).put("text", e.getEntry());
+					events.add(ex);
+				}
+			}
+			JsonNode ex = mapper.createObjectNode();
+			((ObjectNode)ex).put("type", "text");
+			((ObjectNode)ex).put("time", ent.getTimestamp());
+			String txt = ent.toString();
+			((ObjectNode)ex).put("text", txt);
+			events.add(ex);
+		}
+		//	System.out.println("pending messages: "+pendingMessages.size());
+		((ObjectNode)root).put("starttime", starttime);
+		((ObjectNode)root).put("endtime", endtime);
+
+		ArrayNode serversarray = mapper.createArrayNode();
+		((ObjectNode)root).set("servers", serversarray);
+
+		Iterator<Integer> iterator = servers.iterator();
+		while (iterator.hasNext()) {
+			serversarray.add(iterator.next());
+		}
     }
     }
 
 
+    private JsonNode add(String type, long timestamp, int node, String entry){
+		JsonNode stateChange = mapper.createObjectNode();
+		((ObjectNode)stateChange).put("type", type);
+		((ObjectNode)stateChange).put("time", timestamp);
+		((ObjectNode)stateChange).put("server", node);
+		((ObjectNode)stateChange).put("state", entry);
+		return stateChange;
+	}
+
     public String toString() {
     public String toString() {
-	return JSONValue.toJSONString(root);
+		String jsonString = null;
+		try {
+			jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(root);
+		} catch (JsonProcessingException e) {
+			jsonString = "{\"ERR\", " + e.getMessage() + "}";
+		}
+		return jsonString;
     }
     }
 
 
     public static void main(String[] args) throws Exception {
     public static void main(String[] args) throws Exception {
-	MergedLogSource src = new MergedLogSource(args);
-	LogIterator iter = src.iterator();
-	System.out.println(new JsonGenerator(iter));
+		MergedLogSource src = new MergedLogSource(args);
+		LogIterator iter = src.iterator();
+		System.out.println(new JsonGenerator(iter));
     }
     }
 }
 }

+ 125 - 155
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java

@@ -17,48 +17,16 @@
  */
  */
 package org.apache.zookeeper.graph;
 package org.apache.zookeeper.graph;
 
 
-import java.io.ByteArrayInputStream;
-import java.io.EOFException;
-import java.io.FileInputStream;
 import java.io.IOException;
 import java.io.IOException;
-import java.text.DateFormat;
-import java.util.Date;
-import java.util.zip.Adler32;
-import java.util.zip.Checksum;
-import java.util.HashMap;
-
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.InputArchive;
-import org.apache.jute.Record;
-import org.apache.zookeeper.server.TraceFormatter;
-import org.apache.zookeeper.server.persistence.FileHeader;
-import org.apache.zookeeper.server.persistence.FileTxnLog;
-import org.apache.zookeeper.server.util.SerializeUtils;
-import org.apache.zookeeper.txn.TxnHeader;
-
-import org.apache.zookeeper.ZooDefs.OpCode;
-
-import org.apache.zookeeper.txn.CreateSessionTxn;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.ErrorTxn;
-import org.apache.zookeeper.txn.SetACLTxn;
-import org.apache.zookeeper.txn.SetDataTxn;
-import org.apache.zookeeper.txn.TxnHeader;
-
-import java.io.Closeable;
-import java.io.FileNotFoundException;
-import java.util.Vector;
-import java.util.Iterator;
-import java.util.Collections;
-import java.util.NoSuchElementException;
+import java.util.ArrayList;
+import java.util.List;
 
 
 import org.slf4j.Logger;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.slf4j.LoggerFactory;
 
 
 public class MergedLogSource implements LogSource {
 public class MergedLogSource implements LogSource {
     private static final Logger LOG = LoggerFactory.getLogger(MergedLogSource.class);
     private static final Logger LOG = LoggerFactory.getLogger(MergedLogSource.class);
-    private Vector<LogSource> sources = null;
+    protected List<LogSource> sources = new ArrayList<>();
     private long starttime = 0;
     private long starttime = 0;
     private long endtime = 0;
     private long endtime = 0;
     private long size = 0;
     private long size = 0;
@@ -72,148 +40,150 @@ public class MergedLogSource implements LogSource {
     public long getEndTime() { return endtime; }
     public long getEndTime() { return endtime; }
 
 
     private class MergedLogSourceIterator implements LogIterator {
     private class MergedLogSourceIterator implements LogIterator {
-	private LogEntry next = null;
-	private long start = 0;
-	private long end = 0;
-	private MergedLogSource src = null;
-	private LogIterator[] sources = null;
-	private LogEntry[] nexts = null;
-	private FilterOp filter = null;
-	
-	public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
-	    Vector<LogIterator> iters = new Vector<LogIterator>();
-	    for (LogSource s : src.sources) {
-		if (s.overlapsRange(starttime, endtime)) {
-		    iters.add(s.iterator(starttime, endtime, filter));
+		private LogEntry next = null;
+		private long start = 0;
+		private long end = 0;
+		private MergedLogSource src = null;
+		private LogIterator[] sources = null;
+		private LogEntry[] nexts = null;
+		private FilterOp filter = null;
+
+		public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
+			List<LogIterator> iters = new ArrayList<>();
+			for (LogSource s : src.sources) {
+			if (s.overlapsRange(starttime, endtime)) {
+				iters.add(s.iterator(starttime, endtime, filter));
+			}
+			}
+
+			sources = new LogIterator[iters.size()];
+			sources = iters.toArray(sources);
+			nexts = new LogEntry[iters.size()];
+			for (int i = 0; i < sources.length; i++) {
+			if (sources[i].hasNext())
+				nexts[i] = sources[i].next();
+			}
+			this.filter = filter;
 		}
 		}
-	    }
-	    
-	    sources = new LogIterator[iters.size()];
-	    sources = iters.toArray(sources);
-	    nexts = new LogEntry[iters.size()];
-	    for (int i = 0; i < sources.length; i++) {
-		if (sources[i].hasNext()) 
-		    nexts[i] = sources[i].next();
-	    }
-	    this.filter = filter;
-	}
-		    
-	public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime) throws IllegalArgumentException, FilterException {
-	    this(src, starttime, endtime, null);
-	}
-	
-	public long size() throws IOException {
-	    long size = 0;
-	    for (LogIterator i : sources) {
-		size += i.size();
-	    }
-	    return size;
-	}
 
 
-	public boolean hasNext() {
-	    for (LogEntry n : nexts) {
-		if (n != null) return true;
-	    }
-	    return false;
-	}
-	
-	public LogEntry next() {
-	    int min = -1;
-	    for (int i = 0; i < nexts.length; i++) {
-		if (nexts[i] != null) {
-		    if (min == -1) {
-			min = i;
-		    } else if (nexts[i].getTimestamp() < nexts[min].getTimestamp()) {
-			min = i;
-		    }
+		public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime) throws IllegalArgumentException, FilterException {
+			this(src, starttime, endtime, null);
 		}
 		}
-	    }
-	    if (min == -1) {
-		return null;
-	    } else {
-		LogEntry e =  nexts[min];
-		nexts[min] = sources[min].next();
-		return e;
-	    }
-	}
 
 
-	public void remove() throws UnsupportedOperationException {
-	    throw new UnsupportedOperationException("remove not supported for Merged logs");
-	}
-	
-	public void close() throws IOException {
-	    for (LogIterator i : sources) {
-		i.close();
-	    }
-	}
+		public long size() throws IOException {
+			long size = 0;
+			for (LogIterator i : sources) {
+			size += i.size();
+			}
+			return size;
+		}
+
+		public boolean hasNext() {
+			for (LogEntry n : nexts) {
+			if (n != null) return true;
+			}
+			return false;
+		}
+
+		public LogEntry next() {
+			int min = -1;
+			for (int i = 0; i < nexts.length; i++) {
+			if (nexts[i] != null) {
+				if (min == -1) {
+				min = i;
+				} else if (nexts[i].getTimestamp() < nexts[min].getTimestamp()) {
+				min = i;
+				}
+			}
+			}
+			if (min == -1) {
+			return null;
+			} else {
+			LogEntry e =  nexts[min];
+			nexts[min] = sources[min].next();
+			return e;
+			}
+		}
+
+		public void remove() throws UnsupportedOperationException {
+			throw new UnsupportedOperationException("remove not supported for Merged logs");
+		}
+
+		public void close() throws IOException {
+			for (LogIterator i : sources) {
+			i.close();
+			}
+		}
     }
     }
 
 
     public LogIterator iterator(long starttime, long endtime) throws IllegalArgumentException {
     public LogIterator iterator(long starttime, long endtime) throws IllegalArgumentException {
-	try {
-	    return iterator(starttime, endtime, null);
-	} catch (FilterException fe) {
-	    assert(false); // shouldn't happen without filter
-	    return null;
-	}
+		try {
+	    	return iterator(starttime, endtime, null);
+		}
+		catch (FilterException fe) {
+	    	assert(false); // shouldn't happen without filter
+	    	return null;
+		}
     }
     }
 
 
     public LogIterator iterator(long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
     public LogIterator iterator(long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
-	// sanitise start and end times
-	if (endtime < starttime) {
-	    throw new IllegalArgumentException("End time (" +  endtime + ") must be greater or equal to starttime (" + starttime + ")");
-	}
+		// sanitise start and end times
+		if (endtime < starttime) {
+			throw new IllegalArgumentException("End time (" +  endtime + ") must be greater or equal to starttime (" + starttime + ")");
+		}
 
 
-	return new MergedLogSourceIterator(this, starttime, endtime, filter);
-    }
+		return new MergedLogSourceIterator(this, starttime, endtime, filter);
+	}
 
 
-    public LogIterator iterator() throws IllegalArgumentException {
-	return iterator(starttime, endtime+1);
-    }
-    
-    public MergedLogSource(String[] files) throws IOException {
-	sources = new Vector<LogSource>();
-	for (String f : files) {
-	    addSource(f);
+	public LogIterator iterator() throws IllegalArgumentException {
+		return iterator(starttime, endtime+1);
 	}
 	}
+
+	public MergedLogSource(String[] files) throws IOException {
+		sources.clear();
+		for (String f : files) {
+			addSource(f);
+		}
     }
     }
     
     
     public void addSource(String f) throws IOException {
     public void addSource(String f) throws IOException {
-	LogSource s = null;
-	if (TxnLogSource.isTransactionFile(f)) {
-	    s = new TxnLogSource(f);
-	} else {
-	    s = new Log4JSource(f);
-	}
+		LogSource s = null;
+		if (TxnLogSource.isTransactionFile(f)) {
+			s = new TxnLogSource(f);
+		}
+		else {
+			s = new Log4JSource(f);
+		}
 
 
-	size += s.size();
-	endtime = s.getEndTime() > endtime ? s.getEndTime() : endtime;
-	starttime = s.getStartTime() < starttime || starttime == 0 ? s.getStartTime() : starttime;
-	sources.add(s);
+		size += s.size();
+		endtime = s.getEndTime() > endtime ? s.getEndTime() : endtime;
+		starttime = s.getStartTime() < starttime || starttime == 0 ? s.getStartTime() : starttime;
+		sources.add(s);
     }
     }
 
 
     public String toString() {
     public String toString() {
-	String s = "MergedLogSource(size=" + size + ", start=" + starttime + ", end=" + endtime +")";
-	for (LogSource src : sources) {
-	    s += "\n\t- " +src;
-	}
-	return s;
+		String s = "MergedLogSource(size=" + size + ", start=" + starttime + ", end=" + endtime +")";
+		for (LogSource src : sources) {
+			s += "\n\t- " +src;
+		}
+		return s;
     }
     }
 
 
     public static void main(String[] args) throws IOException {
     public static void main(String[] args) throws IOException {
-	System.out.println("Time: " + System.currentTimeMillis());
-	MergedLogSource s = new MergedLogSource(args);
-	System.out.println(s);
-
-	LogIterator iter;
-
-	iter = s.iterator();
-	System.out.println("Time: " + System.currentTimeMillis());
-	System.out.println("Iterator Size: " + iter.size());
-	System.out.println("Time: " + System.currentTimeMillis());
-	/*	while (iter.hasNext()) {
-	    System.out.println(iter.next());
-	    }*/
-	iter.close();
-	System.out.println("Time: " + System.currentTimeMillis());
+		System.out.println("Time: " + System.currentTimeMillis());
+		MergedLogSource s = new MergedLogSource(args);
+		System.out.println(s);
+
+		LogIterator iter;
+
+		iter = s.iterator();
+		System.out.println("Time: " + System.currentTimeMillis());
+		System.out.println("Iterator Size: " + iter.size());
+		System.out.println("Time: " + System.currentTimeMillis());
+		/*	while (iter.hasNext()) {
+			System.out.println(iter.next());
+			}*/
+		iter.close();
+		System.out.println("Time: " + System.currentTimeMillis());
     }
     }
 }
 }

+ 19 - 29
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java

@@ -17,18 +17,10 @@
  */
  */
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 
 
 import org.apache.zookeeper.graph.*;
 import org.apache.zookeeper.graph.*;
 
 
@@ -37,24 +29,22 @@ public class FileLoader extends JsonServlet
     private MergedLogSource source = null;
     private MergedLogSource source = null;
     
     
     public FileLoader(MergedLogSource src) throws Exception {
     public FileLoader(MergedLogSource src) throws Exception {
-	source = src;
+		source = src;
     }
     }
 
 
-    String handleRequest(JsonRequest request) throws Exception
-    {
-	String output = "";
-		
-	String file = request.getString("path", "/");
-	JSONObject o = new JSONObject();
-	try {
-	    this.source.addSource(file);
-	    o.put("status", "OK");
-	
-	} catch (Exception e) {
-	    o.put("status", "ERR");
-	    o.put("error",  e.toString());
-	}
-	
-	return JSONValue.toJSONString(o);
+    String handleRequest(JsonRequest request) throws Exception {
+		String file = request.getString("path", "/");
+		ObjectMapper mapper = new ObjectMapper();
+		JsonNode rootNode = mapper.createObjectNode();
+		try {
+	    	this.source.addSource(file);
+			((ObjectNode) rootNode).put("status", "OK");
+		}
+		catch (Exception e) {
+			((ObjectNode) rootNode).put("status", "ERR");
+			((ObjectNode) rootNode).put("error", e.toString());
+		}
+		String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+		return jsonString;
     }
     }
 }
 }

+ 42 - 38
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java

@@ -18,52 +18,56 @@
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
 import java.io.File;
 import java.io.File;
-import java.io.IOException;
 import java.io.FileNotFoundException;
 import java.io.FileNotFoundException;
 
 
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 
 
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
+import java.io.IOException;
 import java.util.Arrays;
 import java.util.Arrays;
 import java.util.Comparator;
 import java.util.Comparator;
 
 
 public class Fs extends JsonServlet
 public class Fs extends JsonServlet
 {
 {
-    String handleRequest(JsonRequest request) throws Exception
-    {
-	String output = "";
-	JSONArray filelist = new JSONArray();
-
-	File base = new File(request.getString("path", "/"));
-	if (!base.exists() || !base.isDirectory()) {
-	    throw new FileNotFoundException("Couldn't find [" + request + "]");
-	}
-	File[] files = base.listFiles();
-	Arrays.sort(files, new Comparator<File>() { 
-		public int compare(File o1, File o2) {
-		    if (o1.isDirectory() != o2.isDirectory()) {
-			if (o1.isDirectory()) {
-			    return -1;
-			} else {
-			    return 1;
+    String handleRequest(JsonRequest request) throws Exception {
+		File base = new File(request.getString("path", "/"));
+		if (!base.exists() || !base.isDirectory()) {
+			throw new FileNotFoundException("Couldn't find [" + request + "]");
+		}
+		File[] files = base.listFiles();
+		Arrays.sort(files, new Comparator<File>() {
+			public int compare(File o1, File o2) {
+				if (o1.isDirectory() != o2.isDirectory()) {
+				if (o1.isDirectory()) {
+					return -1;
+				} else {
+					return 1;
+				}
+				}
+				return o1.getName().compareToIgnoreCase(o2.getName());
 			}
 			}
-		    }
-		    return o1.getName().compareToIgnoreCase(o2.getName());
-		} 
-	    });
-	
-	for (File f : files) {
-	    JSONObject o = new JSONObject();
-	    o.put("file", f.getName());
-	    o.put("type", f.isDirectory() ? "D" : "F");
-	    o.put("path", f.getCanonicalPath());
-	    filelist.add(o);
-	}
-	return JSONValue.toJSONString(filelist);
+			});
+
+		String jsonString = generateJSON(files);
+		return jsonString;
     }
     }
+
+    protected static String generateJSON(File[] files) throws IOException {
+		ObjectMapper mapper = new ObjectMapper();
+		ArrayNode fileList = mapper.createArrayNode();
+
+		for (File f : files) {
+			JsonNode node = mapper.createObjectNode().objectNode();
+			((ObjectNode) node).put("file", f.getName());
+			((ObjectNode) node).put("type", f.isDirectory() ? "D" : "F");
+			((ObjectNode) node).put("path", f.getCanonicalPath());
+			fileList.add(node);
+		}
+
+		String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(fileList);
+		return jsonString;
+	}
 }
 }

+ 0 - 16
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java

@@ -17,22 +17,6 @@
  */
  */
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import java.util.regex.Pattern;
-import java.util.regex.Matcher;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
 import org.apache.zookeeper.graph.*;
 import org.apache.zookeeper.graph.*;
 import org.slf4j.Logger;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.slf4j.LoggerFactory;

+ 53 - 44
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java

@@ -17,69 +17,78 @@
  */
  */
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 import java.io.IOException;
 import java.io.IOException;
-
 import javax.servlet.ServletException;
 import javax.servlet.ServletException;
 import javax.servlet.ServletRequest;
 import javax.servlet.ServletRequest;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServlet;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 import javax.servlet.http.HttpServletResponse;
 
 
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
 import java.util.Map;
 import java.util.Map;
 
 
 abstract public class JsonServlet extends HttpServlet {
 abstract public class JsonServlet extends HttpServlet {
     abstract String handleRequest(JsonRequest request) throws Exception;
     abstract String handleRequest(JsonRequest request) throws Exception;
 
 
     protected class JsonRequest {
     protected class JsonRequest {
-	private Map map;
+		private Map map;
+
+		public JsonRequest(ServletRequest request) {
+			map = request.getParameterMap();
+		}
+
+		public long getNumber(String name, long defaultnum) {
+			String[] vals = (String[])map.get(name);
+			if (vals == null || vals.length == 0) {
+			return defaultnum;
+			}
 
 
-	public JsonRequest(ServletRequest request) {
-	    map = request.getParameterMap();
-	}
-	
-	public long getNumber(String name, long defaultnum) {
-	    String[] vals = (String[])map.get(name);
-	    if (vals == null || vals.length == 0) {
-		return defaultnum;
-	    }
+			try {
+				return Long.valueOf(vals[0]);
+			}
+			catch (NumberFormatException e) {
+				return defaultnum;
+			}
+		}
 
 
-	    try {
-		return Long.valueOf(vals[0]);
-	    } catch (NumberFormatException e) {
-		return defaultnum;
-	    }
-	}
-	
-	public String getString(String name, String defaultstr) {
-	    String[] vals = (String[])map.get(name);
-	    if (vals == null || vals.length == 0) {
-		return defaultstr;
-	    } else {
-		return vals[0];
-	    }
-	}
+		public String getString(String name, String defaultstr) {
+			String[] vals = (String[])map.get(name);
+			if (vals == null || vals.length == 0) {
+			return defaultstr;
+			}
+			else {
+				return vals[0];
+			}
+		}
     }
     }
 
 
-    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
-    {
+    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
         response.setContentType("text/plain;charset=utf-8");
         response.setContentType("text/plain;charset=utf-8");
         response.setStatus(HttpServletResponse.SC_OK);
         response.setStatus(HttpServletResponse.SC_OK);
-	
-	try {
-	    String req = request.getRequestURI().substring(request.getServletPath().length());
 
 
-	    response.getWriter().println(handleRequest(new JsonRequest(request)));
-	} catch (Exception e) {
-	    JSONObject o = new JSONObject();
-	    o.put("error", e.toString());
-	    response.getWriter().println(JSONValue.toJSONString(o));
-	} catch (java.lang.OutOfMemoryError oom) {
-	    JSONObject o = new JSONObject();
-	    o.put("error", "Out of memory. Perhaps you've requested too many logs. Try narrowing you're filter criteria.");
-	    response.getWriter().println(JSONValue.toJSONString(o));
-	}
+		try {
+			String req = request.getRequestURI().substring(request.getServletPath().length());
+
+			response.getWriter().println(handleRequest(new JsonRequest(request)));
+		}
+		catch (Exception e) {
+			ObjectMapper mapper = new ObjectMapper();
+			JsonNode rootNode = mapper.createObjectNode();
+			((ObjectNode) rootNode).put("error", e.toString());
+			String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+
+			response.getWriter().println(jsonString);
+		}
+		catch (java.lang.OutOfMemoryError oom) {
+			ObjectMapper mapper = new ObjectMapper();
+			JsonNode rootNode = mapper.createObjectNode();
+			((ObjectNode) rootNode).put("error", "Out of memory. Perhaps you've requested too many logs. Try narrowing you're filter criteria.");
+			String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+
+			response.getWriter().println(jsonString);
+		}
     }
     }
 }
 }

+ 39 - 48
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java

@@ -17,27 +17,14 @@
  */
  */
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
-import java.util.regex.Pattern;
-import java.util.regex.Matcher;
-
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 import org.apache.zookeeper.graph.*;
 import org.apache.zookeeper.graph.*;
 import org.slf4j.Logger;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.slf4j.LoggerFactory;
 
 
-
 public class NumEvents extends JsonServlet
 public class NumEvents extends JsonServlet
 {
 {
     private static final Logger LOG = LoggerFactory.getLogger(NumEvents.class);
     private static final Logger LOG = LoggerFactory.getLogger(NumEvents.class);
@@ -46,43 +33,47 @@ public class NumEvents extends JsonServlet
     private LogSource source = null;
     private LogSource source = null;
 
 
     public NumEvents(LogSource src) throws Exception {
     public NumEvents(LogSource src) throws Exception {
-	this.source = src;
+		this.source = src;
     }
     }
 
 
     String handleRequest(JsonRequest request) throws Exception {
     String handleRequest(JsonRequest request) throws Exception {
-	String output = "";
+		String output = "";
+
+		long starttime = 0;
+		long endtime = 0;
+		long period = 0;
+
+		starttime = request.getNumber("start", 0);
+		endtime = request.getNumber("end", 0);
+		period = request.getNumber("period", 0);
+
+		if (starttime == 0) { starttime = source.getStartTime(); }
+		if (endtime == 0) {
+	 	   if (period > 0) {
+				endtime = starttime + period;
+	 	   }
+	 	   else {
+				endtime = source.getEndTime();
+	 	   }
+		}
+
+		long size = 0;
+		LogIterator iter = source.iterator(starttime, endtime);
+		size = iter.size();
 
 
-	long starttime = 0;
-	long endtime = 0;
-	long period = 0;
+		ObjectMapper mapper = new ObjectMapper();
+		JsonNode data = mapper.createObjectNode();
+		((ObjectNode) data).put("startTime", starttime);
+		((ObjectNode) data).put("endTime", endtime);
+		((ObjectNode) data).put("numEntries",  iter.size());
 
 
-	starttime = request.getNumber("start", 0);
-	endtime = request.getNumber("end", 0);
-	period = request.getNumber("period", 0);
+		if (LOG.isDebugEnabled()) {
+		    LOG.debug("handle(start= " + starttime + ", end=" + endtime + ", numEntries=" + size +")");
+		}
+		iter.close();
 
 
-	if (starttime == 0) { starttime = source.getStartTime(); }
-	if (endtime == 0) { 
-	    if (period > 0) {
-		endtime = starttime + period;
-	    } else {
-		endtime = source.getEndTime(); 
-	    }
-	}
-	
-	LogIterator iter = source.iterator(starttime, endtime);
-	JSONObject data = new JSONObject();
-	data.put("startTime", starttime);
-	data.put("endTime", endtime);
-	long size = 0;
-	
-	size = iter.size();
-	
-	data.put("numEntries",  size);
-	if (LOG.isDebugEnabled()) {
-	    LOG.debug("handle(start= " + starttime + ", end=" + endtime + ", numEntries=" + size +")");
-	}
-	iter.close();
-	return JSONValue.toJSONString(data);
+		String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(data);
+		return jsonString;
     }
     }
 }
 }
 
 

+ 85 - 88
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java

@@ -18,23 +18,14 @@
 package org.apache.zookeeper.graph.servlets;
 package org.apache.zookeeper.graph.servlets;
 
 
 import java.io.IOException;
 import java.io.IOException;
-import java.io.BufferedOutputStream;
-import java.io.FileOutputStream;
-import java.io.DataOutputStream;
-import java.io.PrintStream;
-
 import java.util.HashSet;
 import java.util.HashSet;
-import java.util.LinkedHashMap;
 import java.util.Set;
 import java.util.Set;
-
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
 import org.apache.zookeeper.graph.*;
 import org.apache.zookeeper.graph.*;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
 
 
 public class Throughput extends JsonServlet
 public class Throughput extends JsonServlet
 {
 {
@@ -45,82 +36,88 @@ public class Throughput extends JsonServlet
     private LogSource source = null;
     private LogSource source = null;
 
 
     public Throughput(LogSource src) throws Exception {
     public Throughput(LogSource src) throws Exception {
-	this.source = src; 
+		this.source = src;
     }
     }
 
 
     public String handleRequest(JsonRequest request) throws Exception {
     public String handleRequest(JsonRequest request) throws Exception {
-	long starttime = 0;
-	long endtime = 0;
-	long period = 0;
-	long scale = 0;
-	
-	starttime = request.getNumber("start", 0);
-	endtime = request.getNumber("end", 0);
-	period = request.getNumber("period", 0);
-	
-
-	if (starttime == 0) { starttime = source.getStartTime(); }
-	if (endtime == 0) { 
-	    if (period > 0) {
-		endtime = starttime + period;
-	    } else {
-		endtime = source.getEndTime(); 
-	    }
+		long startTime = 0;
+		long endTime = 0;
+		long period = 0;
+		long scale = 0;
+
+		startTime = request.getNumber("start", 0);
+		endTime = request.getNumber("end", 0);
+		period = request.getNumber("period", 0);
+
+
+		if (startTime == 0) { startTime = source.getStartTime(); }
+		if (endTime == 0) {
+			if (period > 0) {
+			endTime = startTime + period;
+			} else {
+			endTime = source.getEndTime();
+			}
+		}
+
+		String scalestr = request.getString("scale", "minutes");
+		if (scalestr.equals("seconds")) {
+			scale = MS_PER_SEC;
+		} else if (scalestr.equals("hours")) {
+			scale = MS_PER_HOUR;
+		} else {
+			scale = MS_PER_MIN;
+		}
+
+		LogIterator iter = source.iterator(startTime, endTime);
+		String jsonString = getJSON(iter, scale);
+		iter.close();
+		return jsonString;
 	}
 	}
-	
-	String scalestr = request.getString("scale", "minutes");
-	if (scalestr.equals("seconds")) {
-	    scale = MS_PER_SEC;
-	} else if (scalestr.equals("hours")) {
-	    scale = MS_PER_HOUR;
-	} else {
-	    scale = MS_PER_MIN;
-	} 	
-	
-	LogIterator iter = source.iterator(starttime, endtime);
-	
-	long current = 0;
-	long currentms = 0;
-	Set<Long> zxids_ms = new HashSet<Long>();
-	long zxidcount = 0;
-
-	JSONArray events = new JSONArray();
-	while (iter.hasNext()) {
-	    LogEntry e = iter.next();
-	    if (e.getType() != LogEntry.Type.TXN) {
-		continue;
-	    }
-
-	    TransactionEntry cxn = (TransactionEntry)e;
-	    
-	    long ms = cxn.getTimestamp();
-	    long inscale = ms/scale;
-
-	    if (currentms != ms && currentms != 0) {
-		zxidcount += zxids_ms.size();
-		zxids_ms.clear();
-	    }
-
-	    if (inscale != current && current != 0) {
-		JSONObject o = new JSONObject();
-		o.put("time", current*scale);
-		o.put("count", zxidcount);
-		events.add(o);
-		zxidcount = 0;
-	    }
-	    current = inscale;
-	    currentms = ms;
-
-	    zxids_ms.add(cxn.getZxid());
-	}
-	JSONObject o = new JSONObject();
-	o.put("time", current*scale);
-	o.put("count", zxidcount);
-	events.add(o);
-
-	iter.close();
-	
-	return JSONValue.toJSONString(events);
-    }
 
 
-};
+	protected String getJSON(final LogIterator iter, final long scale) throws IOException {
+		long current = 0;
+		long currentms = 0;
+		Set<Long> zxids_ms = new HashSet<Long>();
+		long zxidCount = 0;
+
+		ObjectMapper mapper = new ObjectMapper();
+		ArrayNode events = mapper.createArrayNode();
+
+		while (iter.hasNext()) {
+			LogEntry e = iter.next();
+			if (e.getType() != LogEntry.Type.TXN) {
+			continue;
+			}
+
+			TransactionEntry cxn = (TransactionEntry)e;
+
+			long ms = cxn.getTimestamp();
+			long inscale = ms/ scale;
+
+			if (currentms != ms && currentms != 0) {
+				zxidCount += zxids_ms.size();
+				zxids_ms.clear();
+			}
+
+			if (inscale != current && current != 0) {
+				JsonNode node = mapper.createObjectNode();
+				((ObjectNode) node).put("time", current * scale);
+				((ObjectNode) node).put("count", zxidCount);
+				events.add(node);
+				zxidCount = 0;
+			}
+			current = inscale;
+			currentms = ms;
+
+			zxids_ms.add(cxn.getZxid());
+		}
+
+		JsonNode node = mapper.createObjectNode();
+		((ObjectNode) node).put("time", current * scale);
+		((ObjectNode) node).put("count", zxidCount);
+		events.add(node);
+
+		String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(events);
+		return jsonString;
+	}
+}

+ 2 - 7
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh

@@ -20,10 +20,9 @@ make_canonical () {
 }
 }
 
 
 SCRIPTDIR=`dirname $0`
 SCRIPTDIR=`dirname $0`
-BUILDDIR=`make_canonical $SCRIPTDIR/../../../../../build/contrib/loggraph`
-LIBDIR=`make_canonical $BUILDDIR/lib`
+BUILDDIR=`make_canonical $SCRIPTDIR/../../../target/`
+LIBDIR=`make_canonical $BUILDDIR/../lib`
 WEBDIR=`make_canonical $SCRIPTDIR/../web`
 WEBDIR=`make_canonical $SCRIPTDIR/../web`
-ZKDIR=`make_canonical $SCRIPTDIR/../../../../../build/`
 
 
 if [ ! -x $BUILDDIR ]; then
 if [ ! -x $BUILDDIR ]; then
     echo "\n\n*** You need to build loggraph before running it ***\n\n";
     echo "\n\n*** You need to build loggraph before running it ***\n\n";
@@ -34,10 +33,6 @@ for i in `ls $LIBDIR`; do
     CLASSPATH=$LIBDIR/$i:$CLASSPATH
     CLASSPATH=$LIBDIR/$i:$CLASSPATH
 done
 done
 
 
-for i in $ZKDIR/zookeeper-*.jar; do
-    CLASSPATH="$i:$CLASSPATH"
-done
-
 CLASSPATH=$BUILDDIR/classes:$WEBDIR:$CLASSPATH
 CLASSPATH=$BUILDDIR/classes:$WEBDIR:$CLASSPATH
 echo $CLASSPATH
 echo $CLASSPATH
 java -Dlog4j.configuration=org/apache/zookeeper/graph/log4j.properties -Xdebug -Xrunjdwp:transport=dt_socket,address=4444,server=y,suspend=n -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*
 java -Dlog4j.configuration=org/apache/zookeeper/graph/log4j.properties -Xdebug -Xrunjdwp:transport=dt_socket,address=4444,server=y,suspend=n -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*

+ 2 - 7
zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh

@@ -20,9 +20,8 @@ make_canonical () {
 }
 }
 
 
 SCRIPTDIR=`dirname $0`
 SCRIPTDIR=`dirname $0`
-BUILDDIR=`make_canonical $SCRIPTDIR/../../../../../build/contrib/loggraph`
-LIBDIR=`make_canonical $BUILDDIR/lib`
-ZKDIR=`make_canonical $SCRIPTDIR/../../../../../build/`
+BUILDDIR=`make_canonical $SCRIPTDIR/../../../target/`
+LIBDIR=`make_canonical $BUILDDIR/../lib`
 
 
 if [ ! -x $BUILDDIR ]; then
 if [ ! -x $BUILDDIR ]; then
     echo "\n\n*** You need to build loggraph before running it ***\n\n";
     echo "\n\n*** You need to build loggraph before running it ***\n\n";
@@ -37,10 +36,6 @@ for i in `ls $BUILDDIR/*.jar`; do
     CLASSPATH=$i:$CLASSPATH
     CLASSPATH=$i:$CLASSPATH
 done
 done
 
 
-for i in $ZKDIR/zookeeper-*.jar; do
-    CLASSPATH="$i:$CLASSPATH"
-done
-
 java -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*
 java -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*
 
 
 
 

+ 120 - 0
zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FileLoaderTest.java

@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+import org.apache.zookeeper.graph.FilterException;
+import org.apache.zookeeper.graph.FilterOp;
+import org.apache.zookeeper.graph.LogIterator;
+import org.apache.zookeeper.graph.LogSource;
+import org.apache.zookeeper.graph.MergedLogSource;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Assertions;
+import java.io.IOException;
+import java.util.List;
+
+public class FileLoaderTest {
+
+    @Test
+    public void testHandleRequestOK() throws Exception {
+        String[] files = {""};
+        MyMergedLogSource mls = new MyMergedLogSource(files);
+        final JsonServlet.JsonRequest jsonRequest = mock(JsonServlet.JsonRequest.class);
+        when(jsonRequest.getString("path", "/")).thenReturn("/tmp");
+        FileLoader fl = new FileLoader(mls);
+        String s = fl.handleRequest(jsonRequest);
+        Assertions.assertEquals("{\"status\":\"OK\"}", s);
+        Assertions.assertTrue(mls.getSources().contains(new MySource("/tmp")));
+        Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp2")));
+    }
+
+    @Test
+    public void testHandleRequestERR() throws Exception {
+        String[] files = {""};
+        MyMergedLogSource mls = new MyMergedLogSource(files);
+        final JsonServlet.JsonRequest jsonRequest = mock(JsonServlet.JsonRequest.class);
+        when(jsonRequest.getString("path", "/")).thenReturn("/tmp3");
+        FileLoader fl = new FileLoader(mls);
+        String s = fl.handleRequest(jsonRequest);
+        Assertions.assertEquals("{\"status\":\"ERR\",\"error\":\"java.io.IOException: Message\"}", s);
+        Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp")));
+        Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp2")));
+    }
+
+    private class MyMergedLogSource extends MergedLogSource {
+        public MyMergedLogSource(String[] files) throws IOException {
+            super(files);
+        }
+        public void addSource(String f) throws IOException {
+            if ("/tmp3".equals(f)) throw new IOException("Message");
+            sources.add(new MySource(f));
+        }
+        public List<LogSource> getSources(){
+            return sources;
+        }
+    }
+
+    private class MySource implements LogSource{
+        private String file = null;
+        public MySource(String file) throws IOException {
+            this.file=file;
+        }
+
+        public boolean equals(Object o){
+            if(!(o instanceof MySource)) return false;
+            if(((MySource)o).file == null) return this.file==null;
+            return ((MySource)o).file.equals(this.file);
+        }
+
+        @Override
+        public LogIterator iterator(long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
+            return null;
+        }
+
+        @Override
+        public LogIterator iterator(long starttime, long endtime) throws IllegalArgumentException {
+            return null;
+        }
+
+        @Override
+        public LogIterator iterator() throws IllegalArgumentException {
+            return null;
+        }
+
+        @Override
+        public boolean overlapsRange(long starttime, long endtime) {
+            return false;
+        }
+
+        @Override
+        public long size() {
+            return 0;
+        }
+
+        @Override
+        public long getStartTime() {
+            return 0;
+        }
+
+        @Override
+        public long getEndTime() {
+            return 0;
+        }
+    }
+}

+ 46 - 0
zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FsTest.java

@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import java.io.File;
+import java.io.IOException;
+
+public class FsTest {
+    @Test
+    public void testGenerateJSON() throws IOException {
+        File[] files = new File[2];
+        final File file1 = mock(File.class);
+        when(file1.getName()).thenReturn("testDir");
+        when(file1.isDirectory()).thenReturn(true);
+        when(file1.getCanonicalPath()).thenReturn("/tmp/testDir");
+        final File file2 = mock(File.class);
+        when(file2.getName()).thenReturn("test");
+        when(file2.isDirectory()).thenReturn(false);
+        when(file2.getCanonicalPath()).thenReturn("/tmp/test");
+        files[0]=file1;
+        files[1]=file2;
+        String output = Fs.generateJSON(files);
+        String expectedOutput = "[{\"file\":\"testDir\",\"type\":\"D\",\"path\":\"/tmp/testDir\"}," +
+                "{\"file\":\"test\",\"type\":\"F\",\"path\":\"/tmp/test\"}]";
+        Assertions.assertEquals(expectedOutput, output);
+    }
+}

+ 88 - 0
zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/ThroughputTest.java

@@ -0,0 +1,88 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import org.apache.zookeeper.graph.FilterException;
+import org.apache.zookeeper.graph.Log4JEntry;
+import org.apache.zookeeper.graph.Log4JSource;
+import org.apache.zookeeper.graph.LogEntry;
+import org.apache.zookeeper.graph.LogIterator;
+import org.apache.zookeeper.graph.TransactionEntry;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.NoSuchElementException;
+
+public class ThroughputTest {
+
+    @Test
+    public void testGetJSON() throws Exception {
+        long scale = 1;
+        Log4JSource source = mock(Log4JSource.class);
+        Throughput tp = new Throughput(source);
+        LogIterator iter = new MyIterator();
+        String jsonString = tp.getJSON(iter, scale);
+        String expected = "[{\"time\":3,\"count\":1},{\"time\":4,\"count\":1},{\"time\":5,\"count\":0}]";
+        Assertions.assertEquals(expected, jsonString);
+    }
+
+    private class MyIterator implements LogIterator {
+        int index = 0;
+        List<LogEntry> list = new ArrayList<>();
+
+        public MyIterator() throws IllegalArgumentException, FilterException {
+            for(int i=1; i<3; i++){
+                long timestamp = i;
+                int node = i;
+                String entry = Integer.toString(i);
+                Log4JEntry le = new Log4JEntry(timestamp, node, entry);
+                list.add(le);
+            }
+            for(int i=3; i<7; i++){
+                long timestamp = i;
+                long clientId = i;
+                long Cxid = i;
+                long Zxid = i;
+                String op = Integer.toString(i);
+                TransactionEntry te = new TransactionEntry(timestamp, clientId, Cxid, Zxid, op);
+                list.add(te);
+            }
+        }
+
+        synchronized public long size() throws IOException {
+            return list.size();
+        }
+
+        public boolean hasNext() {
+            return index<list.size()-1;
+        }
+
+        public LogEntry next() throws NoSuchElementException {
+            return list.get(index++);
+        }
+
+        public void remove() throws UnsupportedOperationException {
+            throw new UnsupportedOperationException("remove not supported for L4J logs");
+        }
+
+        public void close(){}
+    }
+}

+ 0 - 5
zookeeper-server/pom.xml

@@ -93,11 +93,6 @@
       <artifactId>jackson-databind</artifactId>
       <artifactId>jackson-databind</artifactId>
       <scope>provided</scope>
       <scope>provided</scope>
     </dependency>
     </dependency>
-    <dependency>
-      <groupId>com.googlecode.json-simple</groupId>
-      <artifactId>json-simple</artifactId>
-      <scope>provided</scope>
-    </dependency>
     <dependency>
     <dependency>
       <groupId>org.bouncycastle</groupId>
       <groupId>org.bouncycastle</groupId>
       <artifactId>bcprov-jdk15on</artifactId>
       <artifactId>bcprov-jdk15on</artifactId>

+ 7 - 5
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java

@@ -19,6 +19,7 @@
 package org.apache.zookeeper.server;
 package org.apache.zookeeper.server;
 
 
 import static org.apache.zookeeper.server.persistence.FileSnap.SNAPSHOT_FILE_PREFIX;
 import static org.apache.zookeeper.server.persistence.FileSnap.SNAPSHOT_FILE_PREFIX;
+import com.fasterxml.jackson.core.io.JsonStringEncoder;
 import java.io.File;
 import java.io.File;
 import java.io.IOException;
 import java.io.IOException;
 import java.io.InputStream;
 import java.io.InputStream;
@@ -36,7 +37,6 @@ import org.apache.zookeeper.server.persistence.FileSnap;
 import org.apache.zookeeper.server.persistence.SnapStream;
 import org.apache.zookeeper.server.persistence.SnapStream;
 import org.apache.zookeeper.server.persistence.Util;
 import org.apache.zookeeper.server.persistence.Util;
 import org.apache.zookeeper.util.ServiceUtils;
 import org.apache.zookeeper.util.ServiceUtils;
-import org.json.simple.JSONValue;
 
 
 /**
 /**
  * Dump a snapshot file to stdout.
  * Dump a snapshot file to stdout.
@@ -181,14 +181,16 @@ public class SnapshotFormatter {
     }
     }
 
 
     private void printSnapshotJson(final DataTree dataTree) {
     private void printSnapshotJson(final DataTree dataTree) {
+        JsonStringEncoder encoder = JsonStringEncoder.getInstance();
         System.out.printf(
         System.out.printf(
             "[1,0,{\"progname\":\"SnapshotFormatter.java\",\"progver\":\"0.01\",\"timestamp\":%d}",
             "[1,0,{\"progname\":\"SnapshotFormatter.java\",\"progver\":\"0.01\",\"timestamp\":%d}",
             System.currentTimeMillis());
             System.currentTimeMillis());
-        printZnodeJson(dataTree, "/");
+        printZnodeJson(dataTree, "/", encoder);
         System.out.print("]");
         System.out.print("]");
     }
     }
 
 
-    private void printZnodeJson(final DataTree dataTree, final String fullPath) {
+    private void printZnodeJson(final DataTree dataTree, final String fullPath, JsonStringEncoder encoder) {
+
 
 
         final DataNode n = dataTree.getNode(fullPath);
         final DataNode n = dataTree.getNode(fullPath);
 
 
@@ -209,7 +211,7 @@ public class SnapshotFormatter {
         }
         }
         StringBuilder nodeSB = new StringBuilder();
         StringBuilder nodeSB = new StringBuilder();
         nodeSB.append("{");
         nodeSB.append("{");
-        nodeSB.append("\"name\":\"").append(JSONValue.escape(name)).append("\"").append(",");
+        nodeSB.append("\"name\":\"").append(encoder.quoteAsString(name)).append("\"").append(",");
         nodeSB.append("\"asize\":").append(dataLen).append(",");
         nodeSB.append("\"asize\":").append(dataLen).append(",");
         nodeSB.append("\"dsize\":").append(dataLen).append(",");
         nodeSB.append("\"dsize\":").append(dataLen).append(",");
         nodeSB.append("\"dev\":").append(0).append(",");
         nodeSB.append("\"dev\":").append(0).append(",");
@@ -223,7 +225,7 @@ public class SnapshotFormatter {
         if (children != null && children.size() > 0) {
         if (children != null && children.size() > 0) {
             System.out.print("[" + nodeSB);
             System.out.print("[" + nodeSB);
             for (String child : children) {
             for (String child : children) {
-                printZnodeJson(dataTree, fullPath + (fullPath.equals("/") ? "" : "/") + child);
+                printZnodeJson(dataTree, fullPath + (fullPath.equals("/") ? "" : "/") + child, encoder);
             }
             }
             System.out.print("]");
             System.out.print("]");
         } else {
         } else {

+ 0 - 4
zookeeper-server/src/main/resources/LICENSE.txt

@@ -214,10 +214,6 @@ This distribution bundles SLF4J 1.7.5, which is available under the MIT
 License. For details, see a copy of the license in
 License. For details, see a copy of the license in
 lib/slf4j-1.7.5.LICENSE.txt
 lib/slf4j-1.7.5.LICENSE.txt
 
 
-This distribution bundles json-simple v1.1.1, which is available under the
-Apache Software License, Version 2.0. For details, see a copy of the license in
-lib/json-simple-1.1.1.LICENSE.txt
-
 This distribution bundles a modified version of 'JZLib' as part of
 This distribution bundles a modified version of 'JZLib' as part of
 Netty-3.7.0, which is available under the 3-clause BSD licence. For
 Netty-3.7.0, which is available under the 3-clause BSD licence. For
 details, see a copy of the licence in META-INF/license/LICENSE-jzlib.txt
 details, see a copy of the licence in META-INF/license/LICENSE-jzlib.txt

+ 0 - 202
zookeeper-server/src/main/resources/lib/json-simple-1.1.1.LICENSE.txt

@@ -1,202 +0,0 @@
-
-                                 Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
-
-   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
-   1. Definitions.
-
-      "License" shall mean the terms and conditions for use, reproduction,
-      and distribution as defined by Sections 1 through 9 of this document.
-
-      "Licensor" shall mean the copyright owner or entity authorized by
-      the copyright owner that is granting the License.
-
-      "Legal Entity" shall mean the union of the acting entity and all
-      other entities that control, are controlled by, or are under common
-      control with that entity. For the purposes of this definition,
-      "control" means (i) the power, direct or indirect, to cause the
-      direction or management of such entity, whether by contract or
-      otherwise, or (ii) ownership of fifty percent (50%) or more of the
-      outstanding shares, or (iii) beneficial ownership of such entity.
-
-      "You" (or "Your") shall mean an individual or Legal Entity
-      exercising permissions granted by this License.
-
-      "Source" form shall mean the preferred form for making modifications,
-      including but not limited to software source code, documentation
-      source, and configuration files.
-
-      "Object" form shall mean any form resulting from mechanical
-      transformation or translation of a Source form, including but
-      not limited to compiled object code, generated documentation,
-      and conversions to other media types.
-
-      "Work" shall mean the work of authorship, whether in Source or
-      Object form, made available under the License, as indicated by a
-      copyright notice that is included in or attached to the work
-      (an example is provided in the Appendix below).
-
-      "Derivative Works" shall mean any work, whether in Source or Object
-      form, that is based on (or derived from) the Work and for which the
-      editorial revisions, annotations, elaborations, or other modifications
-      represent, as a whole, an original work of authorship. For the purposes
-      of this License, Derivative Works shall not include works that remain
-      separable from, or merely link (or bind by name) to the interfaces of,
-      the Work and Derivative Works thereof.
-
-      "Contribution" shall mean any work of authorship, including
-      the original version of the Work and any modifications or additions
-      to that Work or Derivative Works thereof, that is intentionally
-      submitted to Licensor for inclusion in the Work by the copyright owner
-      or by an individual or Legal Entity authorized to submit on behalf of
-      the copyright owner. For the purposes of this definition, "submitted"
-      means any form of electronic, verbal, or written communication sent
-      to the Licensor or its representatives, including but not limited to
-      communication on electronic mailing lists, source code control systems,
-      and issue tracking systems that are managed by, or on behalf of, the
-      Licensor for the purpose of discussing and improving the Work, but
-      excluding communication that is conspicuously marked or otherwise
-      designated in writing by the copyright owner as "Not a Contribution."
-
-      "Contributor" shall mean Licensor and any individual or Legal Entity
-      on behalf of whom a Contribution has been received by Licensor and
-      subsequently incorporated within the Work.
-
-   2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
-
-   3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
-      by such Contributor that are necessarily infringed by their
-      Contribution(s) alone or by combination of their Contribution(s)
-      with the Work to which such Contribution(s) was submitted. If You
-      institute patent litigation against any entity (including a
-      cross-claim or counterclaim in a lawsuit) alleging that the Work
-      or a Contribution incorporated within the Work constitutes direct
-      or contributory patent infringement, then any patent licenses
-      granted to You under this License for that Work shall terminate
-      as of the date such litigation is filed.
-
-   4. Redistribution. You may reproduce and distribute copies of the
-      Work or Derivative Works thereof in any medium, with or without
-      modifications, and in Source or Object form, provided that You
-      meet the following conditions:
-
-      (a) You must give any other recipients of the Work or
-          Derivative Works a copy of this License; and
-
-      (b) You must cause any modified files to carry prominent notices
-          stating that You changed the files; and
-
-      (c) You must retain, in the Source form of any Derivative Works
-          that You distribute, all copyright, patent, trademark, and
-          attribution notices from the Source form of the Work,
-          excluding those notices that do not pertain to any part of
-          the Derivative Works; and
-
-      (d) If the Work includes a "NOTICE" text file as part of its
-          distribution, then any Derivative Works that You distribute must
-          include a readable copy of the attribution notices contained
-          within such NOTICE file, excluding those notices that do not
-          pertain to any part of the Derivative Works, in at least one
-          of the following places: within a NOTICE text file distributed
-          as part of the Derivative Works; within the Source form or
-          documentation, if provided along with the Derivative Works; or,
-          within a display generated by the Derivative Works, if and
-          wherever such third-party notices normally appear. The contents
-          of the NOTICE file are for informational purposes only and
-          do not modify the License. You may add Your own attribution
-          notices within Derivative Works that You distribute, alongside
-          or as an addendum to the NOTICE text from the Work, provided
-          that such additional attribution notices cannot be construed
-          as modifying the License.
-
-      You may add Your own copyright statement to Your modifications and
-      may provide additional or different license terms and conditions
-      for use, reproduction, or distribution of Your modifications, or
-      for any such Derivative Works as a whole, provided Your use,
-      reproduction, and distribution of the Work otherwise complies with
-      the conditions stated in this License.
-
-   5. Submission of Contributions. Unless You explicitly state otherwise,
-      any Contribution intentionally submitted for inclusion in the Work
-      by You to the Licensor shall be under the terms and conditions of
-      this License, without any additional terms or conditions.
-      Notwithstanding the above, nothing herein shall supersede or modify
-      the terms of any separate license agreement you may have executed
-      with Licensor regarding such Contributions.
-
-   6. Trademarks. This License does not grant permission to use the trade
-      names, trademarks, service marks, or product names of the Licensor,
-      except as required for reasonable and customary use in describing the
-      origin of the Work and reproducing the content of the NOTICE file.
-
-   7. Disclaimer of Warranty. Unless required by applicable law or
-      agreed to in writing, Licensor provides the Work (and each
-      Contributor provides its Contributions) on an "AS IS" BASIS,
-      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-      implied, including, without limitation, any warranties or conditions
-      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
-      PARTICULAR PURPOSE. You are solely responsible for determining the
-      appropriateness of using or redistributing the Work and assume any
-      risks associated with Your exercise of permissions under this License.
-
-   8. Limitation of Liability. In no event and under no legal theory,
-      whether in tort (including negligence), contract, or otherwise,
-      unless required by applicable law (such as deliberate and grossly
-      negligent acts) or agreed to in writing, shall any Contributor be
-      liable to You for damages, including any direct, indirect, special,
-      incidental, or consequential damages of any character arising as a
-      result of this License or out of the use or inability to use the
-      Work (including but not limited to damages for loss of goodwill,
-      work stoppage, computer failure or malfunction, or any and all
-      other commercial damages or losses), even if such Contributor
-      has been advised of the possibility of such damages.
-
-   9. Accepting Warranty or Additional Liability. While redistributing
-      the Work or Derivative Works thereof, You may choose to offer,
-      and charge a fee for, acceptance of support, warranty, indemnity,
-      or other liability obligations and/or rights consistent with this
-      License. However, in accepting such obligations, You may act only
-      on Your own behalf and on Your sole responsibility, not on behalf
-      of any other Contributor, and only if You agree to indemnify,
-      defend, and hold each Contributor harmless for any liability
-      incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
-
-   END OF TERMS AND CONDITIONS
-
-   APPENDIX: How to apply the Apache License to your work.
-
-      To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "[]"
-      replaced with your own identifying information. (Don't include
-      the brackets!)  The text should be enclosed in the appropriate
-      comment syntax for the file format. We also recommend that a
-      file or class name and description of purpose be included on the
-      same "printed page" as the copyright notice for easier
-      identification within third-party archives.
-
-   Copyright 1999-2005 The Apache Software Foundation
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
-
-       http://www.apache.org/licenses/LICENSE-2.0
-
-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.