Upgrade to protobuf 2.5 (and how to work with Protobuf)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view

Upgrade to protobuf 2.5 (and how to work with Protobuf)

Edson Tirelli-4


   Today protobuf used by Drools/jBPM was upgrade to version 2.5.0. If you only use the marshalling code, you don't need to do anything. Maven should automatically download the new version and use it.

   If you do protobuf development and need to re-generate the protobuf classes from time to time, here is a reminder of how to configure your environment. Please note that the configuration of the plugin (specially for the jBPM module) is extremely important, or you will start to have all kinds of errors and will not know why.

1. Download and install protobuf compiler from: http://code.google.com/p/protobuf/downloads/list

2. Install protobuf-dt eclipse plugin as detailed here: http://code.google.com/p/protobuf-dt/

3. Configure the destination directory to generate the java files as detailed in step 2 here:

Set the "Java Output Directory" to: src/main/java

4. VERY IMPORTANT FOR JBPM: jBPM Flow project has a .proto file that extends the data types defined in the .proto file in Drools Core, so you need to configure the protobuf import path as detailed here:

Add the following import paths to the jbpm-flow project:

${your drools-core project in eclipse}/src/main/resources

   After doing that, when you edit and save the .proto files, eclipse will automatically compile and regenerate the java source code for them.-- 


  Edson Tirelli
  JBoss Drools Core Development
  JBoss by Red Hat @ www.jboss.com
rules-dev mailing list
[hidden email]
Reply | Threaded
Open this post in threaded view

Re: Upgrade to protobuf 2.5 (and how to work with Protobuf)

This post has NOT been accepted by the mailing list yet.
It looks like moving from protobuf-java v.2.4.1 to v.2.5.0 has non-passive changes that are causing more troubles preventing an upgrade to Drools v.6.x.

Explained here

Our Drools rules are ran in an environment that must share a classpath with Hadoop libs.  These libs are still using protobuf-java v.2.4.1 and cannot easily be upgraded.

The problem comes down to a runtime error:
2014-03-11 06:39:25,229 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.VerifyError: class org.drools.compiler.kie.builder.impl.KieModuleCache$KModuleCache overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
  at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at org.drools.compiler.kie.builder.impl.KieBuilderImpl.createCacheBuilder(KieBuilderImpl.java:269)
        at org.drools.compiler.kie.builder.impl.KieBuilderImpl.generateKieModuleMetaInfo(KieBuilderImpl.java:224)
        at org.drools.compiler.kie.builder.impl.KieBuilderImpl.buildAll(KieBuilderImpl.java:194)
        <... application level omitted ...>

This error makes sense given the changes in protobuf-java v.2.5.

I do not believe that our use-case of the Drools rules engine involves the use of any of the features of the `KieModuleCache` and marshalling/unmarshalling libs associated with it.  
However, I do not see any sort of configuration that would avoid this error.

I tried to simply use the "legacy" Drools knowledge-api when upgrading to Drools v.6.x, but this has failed us since there are several unimplemented methods in the `org.drools.impl.adapters.KnowledgeRuntimeAdapter`, such as `org.drools.impl.adapters.KnowledgeRuntimeAdapter#getQueryResults`.

Side note:  I expected the knowledge-api to be fully-functional and implemented in Drools v6.x for backwards compatibility and for tooling integration.
However, this does not seem to be the case at this point.

We are eager to move to Drools v6.x to avoid some performance issues we are facing due to performance issues with  eagerly evaluating `AccumulateNode` results that are accumulating large collections.

Do you have any suggestions?