RSS

Category Archives: JBoss Fuse

JBoss Fuse Jolokia requests

Even though Hawtio is a pretty awesome console for JBoss Fuse, sometimes you want or even need to have the underlying data Hawtio uses. Maybe you want to incorporate it in your existing dashboard, maybe you want to store it in a database so you have a historical record, or maybe you’re just curious and want to experiment with it.

The nice thing of JBoss Fuse and really the underlying community products, mainly Apache Camel and Apache ActiveMQ, is that they expose a lot of metrics and statistics via JMX. Now you could start your favorite IDE or text editor and write a slab of Java code to query the JMX metrics from Apache Camel and Apache ActiveMQ, like some sort of maniac. Or you could leverage the out of the box Jolokia features, like the people from Hawtio did, and really any sane person would do.

For those of you who never heard of Jolokia, in a nutshell its basically JMX over http/json. Which makes doing JMX almost fun again.

Now to start things off here are a couple Jolokia requests (and responses) to start you off. These are just plain http requests, do note they use basic authentication to authenticate against the JBoss Fuse server.

To start things of some statistics on the JVM:

The request for querying the heapsize of the JVM JBoss Fuse is running on:

http://localhost:8181/hawtio/jolokia/read/java.lang:type=Memory/HeapMemoryUsage

Which returns, on my local test machine, the following response:

{
   "request":{
      "mbean":"java.lang:type=Memory",
      "attribute":"HeapMemoryUsage",
      "type":"read"
   },
   "value":{
      "init":536870912,
      "committed":749207552,
      "max":954728448,
      "used":118298056
   },
   "timestamp":1469035168,
   "status":200
}

Now we know how to query the Heap memory, fetching the NonHeap is pretty trivial:

http://localhost:8181/hawtio/jolokia/read/java.lang:type=Memory/NonHeapMemoryUsage

{
   "request":{
      "mbean":"java.lang:type=Memory",
      "attribute":"NonHeapMemoryUsage",
      "type":"read"
   },
   "value":{
      "init":2555904,
      "committed":120848384,
      "max":-1,
      "used":110152704
   },
   "timestamp":1469035298,
   "status":200
}

Some additional threading statistics for the JVM:

http://localhost:8181/hawtio/jolokia/read/java.lang:type=Threading

which returns:

{
   "request":{
      "mbean":"java.lang:type=Threading",
      "type":"read"
   },
   "value":{
      "ThreadAllocatedMemorySupported":true,
      "ThreadContentionMonitoringEnabled":false,
      "TotalStartedThreadCount":125,
      "CurrentThreadCpuTimeSupported":true,
      "CurrentThreadUserTime":610000000,
      "PeakThreadCount":108,
      "AllThreadIds":[
         150,
         45,
         123,
         122,
         121,
         120,
         119,
         118,
         117,
         116,
         115,
         113,
         111,
         110,
         109,
         107,
         106,
         105,
         104,
         103,
         102,
         101,
         100,
         99,
         98,
         97,
         96,
         95,
         94,
         93,
         92,
         90,
         87,
         86,
         84,
         83,
         82,
         81,
         80,
         79,
         78,
         77,
         76,
         74,
         73,
         72,
         71,
         70,
         69,
         68,
         67,
         65,
         62,
         61,
         60,
         58,
         57,
         55,
         54,
         53,
         51,
         47,
         46,
         44,
         42,
         40,
         39,
         38,
         37,
         36,
         35,
         34,
         33,
         32,
         31,
         30,
         28,
         27,
         26,
         24,
         25,
         21,
         20,
         23,
         22,
         19,
         18,
         15,
         14,
         13,
         12,
         11,
         4,
         3,
         2,
         1
      ],
      "ThreadAllocatedMemoryEnabled":true,
      "CurrentThreadCpuTime":624340113,
      "ObjectName":{
         "objectName":"java.lang:type=Threading"
      },
      "ThreadContentionMonitoringSupported":true,
      "ThreadCpuTimeSupported":true,
      "ThreadCount":96,
      "ThreadCpuTimeEnabled":true,
      "ObjectMonitorUsageSupported":true,
      "SynchronizerUsageSupported":true,
      "DaemonThreadCount":70
   },
   "timestamp":1469035389,
   "status":200
}

Now for some JBoss Fuse specific queries:

First some overall statistics for our ActiveMQ broker:

http://localhost:8181/hawtio/jolokia/read/org.apache.activemq:type=Broker,brokerName=amq

Which returns:

{
   "request":{
      "mbean":"org.apache.activemq:brokerName=amq,type=Broker",
      "type":"read"
   },
   "value":{
      "StatisticsEnabled":true,
      "TotalConnectionsCount":0,
      "StompSslURL":"",
      "TransportConnectors":{
         "openwire":"tcp:\/\/localhost:61616",
         "stomp":"stomp+nio:\/\/pim-XPS-15-9530:61613"
      },
      "StompURL":"",
      "TotalProducerCount":0,
      "CurrentConnectionsCount":0,
      "TopicProducers":[

      ],
      "JMSJobScheduler":null,
      "UptimeMillis":461929,
      "TemporaryQueueProducers":[

      ],
      "TotalDequeueCount":0,
      "JobSchedulerStorePercentUsage":0,
      "DurableTopicSubscribers":[

      ],
      "QueueSubscribers":[

      ],
      "AverageMessageSize":1024,
      "BrokerVersion":"5.11.0.redhat-621084",
      "TemporaryQueues":[

      ],
      "BrokerName":"amq",
      "MinMessageSize":1024,
      "DynamicDestinationProducers":[

      ],
      "OpenWireURL":"tcp:\/\/localhost:61616",
      "TemporaryTopics":[

      ],
      "JobSchedulerStoreLimit":0,
      "TotalConsumerCount":0,
      "MaxMessageSize":1024,
      "TotalMessageCount":0,
      "TempPercentUsage":0,
      "TemporaryQueueSubscribers":[

      ],
      "MemoryPercentUsage":0,
      "SslURL":"",
      "InactiveDurableTopicSubscribers":[

      ],
      "StoreLimit":10737418240,
      "QueueProducers":[

      ],
      "VMURL":"vm:\/\/amq",
      "TemporaryTopicProducers":[

      ],
      "Topics":[
         {
            "objectName":"org.apache.activemq:brokerName=amq,destinationName=ActiveMQ.Advisory.MasterBroker,destinationType=Topic,type=Broker"
         }
      ],
      "Uptime":"7 minutes",
      "BrokerId":"ID:pim-XPS-15-9530-35535-1469035012806-0:1",
      "DataDirectory":"\/home\/pim\/apps\/jboss-fuse-6.2.1.redhat-084\/data\/amq",
      "Persistent":true,
      "TopicSubscribers":[

      ],
      "MemoryLimit":668309914,
      "Slave":false,
      "TotalEnqueueCount":1,
      "TempLimit":5368709120,
      "TemporaryTopicSubscribers":[

      ],
      "StorePercentUsage":0,
      "Queues":[

      ]
   },
   "timestamp":1469035474,
   "status":200
}

We can also zoom into a specific JMS destination on our broker, in this case the Queue named testQueue:

http://localhost:8181/hawtio/jolokia/read/org.apache.activemq:type=Broker,brokerName=amq,destinationType=Queue,destinationName=testQueue

{
   "request":{
      "mbean":"org.apache.activemq:brokerName=amq,destinationName=testQueue,destinationType=Queue,type=Broker",
      "type":"read"
   },
   "value":{
      "ProducerFlowControl":true,
      "AlwaysRetroactive":false,
      "Options":"",
      "MemoryUsageByteCount":0,
      "AverageBlockedTime":0.0,
      "MemoryPercentUsage":0,
      "CursorMemoryUsage":0,
      "InFlightCount":0,
      "Subscriptions":[

      ],
      "CacheEnabled":true,
      "ForwardCount":0,
      "DLQ":false,
      "AverageEnqueueTime":0.0,
      "Name":"testQueue",
      "MaxAuditDepth":10000,
      "BlockedSends":0,
      "TotalBlockedTime":0,
      "MaxPageSize":200,
      "QueueSize":0,
      "PrioritizedMessages":false,
      "MemoryUsagePortion":0.0,
      "Paused":false,
      "EnqueueCount":0,
      "MessageGroups":{

      },
      "ConsumerCount":0,
      "AverageMessageSize":0,
      "CursorFull":false,
      "MaxProducersToAudit":64,
      "ExpiredCount":0,
      "CursorPercentUsage":0,
      "MinEnqueueTime":0,
      "MinMessageSize":0,
      "MemoryLimit":1048576,
      "DispatchCount":0,
      "MaxEnqueueTime":0,
      "BlockedProducerWarningInterval":30000,
      "DequeueCount":0,
      "ProducerCount":0,
      "MessageGroupType":"cached",
      "MaxMessageSize":0,
      "UseCache":true,
      "SlowConsumerStrategy":null
   },
   "timestamp":1469035582,
   "status":200
}

And next to ActiveMQ we can also retrieve some Camel statistics:

http://localhost:8181/hawtio/jolokia/read/org.apache.camel:context=*,type=routes,name=*/Load01,InflightExchanges,LastProcessingTime,ExchangesFailed

Which returns the metrics: “Load01”, “InflightExchanges”, “LastProcessingTime” and “ExchangesFailed” from all Camel routes in all Camel Contexts:

{
   "request":{
      "mbean":"org.apache.camel:context=*,name=*,type=routes",
      "attribute":[
         "Load01",
         "InflightExchanges",
         "LastProcessingTime",
         "ExchangesFailed"
      ],
      "type":"read"
   },
   "value":{
      "org.apache.camel:context=camel-blueprint,name=\"route1\",type=routes":{
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "InflightExchanges":0,
         "Load01":""
      },
      "org.apache.camel:context=camel-blueprint,name=\"route3\",type=routes":{
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "InflightExchanges":0,
         "Load01":""
      },
      "org.apache.camel:context=camel-blueprint,name=\"route2\",type=routes":{
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "InflightExchanges":0,
         "Load01":""
      }
   },
   "timestamp":1469035650,
   "status":200
}

We can off course also remove some filters to get all available statisitics on the Camel routes:

http://localhost:8181/hawtio/jolokia/read/org.apache.camel:context=*,type=routes,name=*

which returns a lot more statistics:

{
   "request":{
      "mbean":"org.apache.camel:context=*,name=*,type=routes",
      "type":"read"
   },
   "value":{
      "org.apache.camel:context=camel-blueprint,name=\"route1\",type=routes":{
         "StatisticsEnabled":true,
         "EndpointUri":"rest:\/\/get:\/user:\/%7Bid%7D?description=Find+user+by+id&outType=nl.schiphol.my.camel.swagger.example.User&routeId=route1",
         "CamelManagementName":"camel-blueprint",
         "ExchangesCompleted":0,
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "Description":null,
         "FirstExchangeCompletedExchangeId":null,
         "FirstExchangeCompletedTimestamp":null,
         "LastExchangeFailureTimestamp":null,
         "MaxProcessingTime":0,
         "LastExchangeCompletedTimestamp":null,
         "Load15":"",
         "DeltaProcessingTime":0,
         "OldestInflightDuration":null,
         "ExternalRedeliveries":0,
         "ExchangesTotal":0,
         "ResetTimestamp":"2016-07-20T19:16:53+02:00",
         "ExchangesInflight":0,
         "MeanProcessingTime":0,
         "LastExchangeFailureExchangeId":null,
         "FirstExchangeFailureExchangeId":null,
         "CamelId":"myCamel",
         "TotalProcessingTime":0,
         "FirstExchangeFailureTimestamp":null,
         "RouteId":"route1",
         "RoutePolicyList":"",
         "FailuresHandled":0,
         "MessageHistory":true,
         "Load05":"",
         "OldestInflightExchangeId":null,
         "State":"Started",
         "InflightExchanges":0,
         "Redeliveries":0,
         "MinProcessingTime":0,
         "LastExchangeCompletedExchangeId":null,
         "Tracing":false,
         "Load01":""
      },
      "org.apache.camel:context=camel-blueprint,name=\"route3\",type=routes":{
         "StatisticsEnabled":true,
         "EndpointUri":"rest:\/\/get:\/user:\/findAll?description=Find+all+users&outType=nl.schiphol.my.camel.swagger.example.User%5B%5D&routeId=route3",
         "CamelManagementName":"camel-blueprint",
         "ExchangesCompleted":0,
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "Description":null,
         "FirstExchangeCompletedExchangeId":null,
         "FirstExchangeCompletedTimestamp":null,
         "LastExchangeFailureTimestamp":null,
         "MaxProcessingTime":0,
         "LastExchangeCompletedTimestamp":null,
         "Load15":"",
         "DeltaProcessingTime":0,
         "OldestInflightDuration":null,
         "ExternalRedeliveries":0,
         "ExchangesTotal":0,
         "ResetTimestamp":"2016-07-20T19:16:53+02:00",
         "ExchangesInflight":0,
         "MeanProcessingTime":0,
         "LastExchangeFailureExchangeId":null,
         "FirstExchangeFailureExchangeId":null,
         "CamelId":"myCamel",
         "TotalProcessingTime":0,
         "FirstExchangeFailureTimestamp":null,
         "RouteId":"route3",
         "RoutePolicyList":"",
         "FailuresHandled":0,
         "MessageHistory":true,
         "Load05":"",
         "OldestInflightExchangeId":null,
         "State":"Started",
         "InflightExchanges":0,
         "Redeliveries":0,
         "MinProcessingTime":0,
         "LastExchangeCompletedExchangeId":null,
         "Tracing":false,
         "Load01":""
      },
      "org.apache.camel:context=camel-blueprint,name=\"route2\",type=routes":{
         "StatisticsEnabled":true,
         "EndpointUri":"rest:\/\/put:\/user?description=Updates+or+create+a+user&inType=nl.schiphol.my.camel.swagger.example.User&routeId=route2",
         "CamelManagementName":"camel-blueprint",
         "ExchangesCompleted":0,
         "LastProcessingTime":0,
         "ExchangesFailed":0,
         "Description":null,
         "FirstExchangeCompletedExchangeId":null,
         "FirstExchangeCompletedTimestamp":null,
         "LastExchangeFailureTimestamp":null,
         "MaxProcessingTime":0,
         "LastExchangeCompletedTimestamp":null,
         "Load15":"",
         "DeltaProcessingTime":0,
         "OldestInflightDuration":null,
         "ExternalRedeliveries":0,
         "ExchangesTotal":0,
         "ResetTimestamp":"2016-07-20T19:16:53+02:00",
         "ExchangesInflight":0,
         "MeanProcessingTime":0,
         "LastExchangeFailureExchangeId":null,
         "FirstExchangeFailureExchangeId":null,
         "CamelId":"myCamel",
         "TotalProcessingTime":0,
         "FirstExchangeFailureTimestamp":null,
         "RouteId":"route2",
         "RoutePolicyList":"",
         "FailuresHandled":0,
         "MessageHistory":true,
         "Load05":"",
         "OldestInflightExchangeId":null,
         "State":"Started",
         "InflightExchanges":0,
         "Redeliveries":0,
         "MinProcessingTime":0,
         "LastExchangeCompletedExchangeId":null,
         "Tracing":false,
         "Load01":""
      }
   },
   "timestamp":1469035760,
   "status":200
}

Now there are probably tens or hundreds more queries to think of, but hopefully this will start you off in the right direction.

Advertisements
 
Leave a comment

Posted by on 2016-07-20 in JBoss Fuse

 

Tags: , , , , , ,

Instantiating a java.util.Properties via OSGi Blueprint

I was recently struggling a bit to instantiate a java.util.Properties object via OSGi Blueprint. Normally when dealing with properties in OSGi Blueprint I use the OSGi Admin config or Compendium service. However one of the other objects I needed required a java.util.Properties object as a constructor argument. So I needed to instantiate it via Blueprint. After some try & error I found the solution.


<bean id="myPropertiesObject" class="java.util.Properties">
    <argument>
      <props>
        <prop key="property1">value1</prop>
        <prop key="property2">value1</prop>
        <prop key="property3">value1</prop>
      </props>
    </argument>
  </bean>
 
2 Comments

Posted by on 2016-03-28 in Geen categorie, JBoss Fuse

 

Tags: , ,

MVN camel:run exception in Fuse 6.2.1

When playing around with a new Fuse 6.2.1 environment I notices the maven archetypes provided with Fuse did not run out of the box with the mvn camel:run command.

Whenever I executed the command I received the following error:


[ERROR] Failed to execute goal org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084:run (default-cli) on project postback-core: Execution default-cli of goal org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084:run failed: Plugin org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.camel:camel-maven-plugin:jar:2.15.1.redhat-621084 -> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.0 -> org.apache.velocity:velocity:jar:1.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7: Failed to read artifact descriptor for commons-collections:commons-collections:jar:3.2.1.redhat-7: Failure to find org.apache.commons:commons-parent:pom:22-redhat-2 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

Executing the command with the –e flag gave the following stacktrace:


org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084:run (default-cli) on project postback-core: Execution default-cli of goal org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084:run failed: Plugin org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.camel:camel-maven-plugin:jar:2.15.1.redhat-621084 -> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.0 -> org.apache.velocity:velocity:jar:1.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)

at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)

at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084:run failed: Plugin org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.camel:camel-maven-plugin:jar:2.15.1.redhat-621084 -> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.0 -> org.apache.velocity:velocity:jar:1.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)

at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)

... 20 more

Caused by: org.apache.maven.plugin.PluginResolutionException: Plugin org.apache.camel:camel-maven-plugin:2.15.1.redhat-621084 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.camel:camel-maven-plugin:jar:2.15.1.redhat-621084 -> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.0 -> org.apache.velocity:velocity:jar:1.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7

at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:214)

at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve(DefaultPluginDependenciesResolver.java:149)

at org.apache.maven.plugin.internal.DefaultMavenPluginManager.createPluginRealm(DefaultMavenPluginManager.java:400)

at org.apache.maven.plugin.internal.DefaultMavenPluginManager.setupPluginRealm(DefaultMavenPluginManager.java:372)

at org.apache.maven.plugin.DefaultBuildPluginManager.getPluginRealm(DefaultBuildPluginManager.java:231)

at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:102)

... 21 more

Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at org.apache.camel:camel-maven-plugin:jar:2.15.1.redhat-621084 -> org.apache.maven.reporting:maven-reporting-impl:jar:2.0.5 -> org.apache.maven.doxia:doxia-site-renderer:jar:1.0 -> org.apache.velocity:velocity:jar:1.5 -> commons-collections:commons-collections:jar:3.2.1.redhat-7

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:316)

at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolveInternal(DefaultPluginDependenciesResolver.java:202)

... 26 more

Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for commons-collections:commons-collections:jar:3.2.1.redhat-7

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:329)

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:198)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:535)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:519)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:409)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:504)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:504)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:504)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)

... 28 more

Caused by: org.apache.maven.model.resolution.UnresolvableModelException: Failure to find org.apache.commons:commons-parent:pom:22-redhat-2 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:177)

at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:226)

at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:1000)

at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:800)

at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:329)

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320)

... 47 more

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Failure to find org.apache.commons:commons-parent:pom:22-redhat-2 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223)

at org.apache.maven.repository.internal.DefaultModelResolver.resolveModel(DefaultModelResolver.java:173)

... 52 more

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Failure to find org.apache.commons:commons-parent:pom:22-redhat-2 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:231)

at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:206)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.gatherDownloads(DefaultArtifactResolver.java:585)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:503)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)

... 55 more

It took me some time to get this resolved in the end the solution was configuring another maven plugin repository:


<pluginRepository>
  <id>redhat</id>
  <url>https://maven.repository.redhat.com/ga/</url>
  <releases>
    <enabled>true</enabled>
  </releases>
</pluginRepository>

 
1 Comment

Posted by on 2016-01-31 in JBoss Fuse

 

Tags: , ,

Using a custom BundleActivator class in your Camel OSGi bundle

For those not familiar with the BundleActivator class in OSGi it is used to control the startup and shutdown of an OSGi bundle. From the OSGI wiki:

“Bundle-Activator is a Manifest Header which specifies a class, implementing the org.osgi.framework.BundleActivator interface, which will be called at bundle activation time and deactivation time.”

When implementing a Camel context and deploying it as an OSGi bundle (for example when using JBoss Fuse or Apache Servicemix) no custom bundleactivator is used. In this post I will explain how to create a custom BundleActivator and add to the manifest of the OSGi bundle.

Creating the custom BundleActivator class

As the quote from above states we need to implement the BundleActivator interface. We create a new Java class implementing this interface.

When we implement the interface we need to implement two methods. The auto-generated method stubs created by Eclipse look like this:


package nl.rubix.custombundleactovator;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class MyBundleActivator implements BundleActivator{

    @Override
    public void start(BundleContext arg0) throws Exception {
        // TODO Auto-generated method stub
        
    }

    @Override
    public void stop(BundleContext arg0) throws Exception {
        // TODO Auto-generated method stub
        
    }

}

The methods are quite self-explanatory, the start method is called when the OSGi bundle is started, the stop method when the bundle is stopped 🙂

To demonstrate the BundleActivator some simple logging is added so we have something to look at.


package nl.rubix.custombundleactovator;

import org.apache.log4j.Logger;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class MyBundleActivator implements BundleActivator{

    private Logger logger = Logger.getLogger(this.getClass());
    
    @Override
    public void start(BundleContext arg0) throws Exception {
        logger.info("look ma im here.....");
    }

    @Override
    public void stop(BundleContext arg0) throws Exception {
        logger.info("screw you guys im going home...");
    }

}

Adding the custom BundleActivator to the OSGi bundle

To add the custom BundleActivator to the OSGi bundle we can use the maven felix plugin. When using the JBoss Fuse packaged product from Red Hat it is actually a best practice to use the maven plugin for creating the OSGi manifest. https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.2/html/Deploying_into_the_Container/BestPractices.html#BestPractices-Tooling-UTMBPTGTM

To add the custom BundleActivator class add the Bundle-Activator option to the configuration instructions and point to the class.


<!-- to generate the MANIFEST-FILE of the bundle -->
<plugin>
<groupId>org.apache.felix</groupId>
<artifactId>maven-bundle-plugin</artifactId>
<version>2.3.7</version>
<extensions>true</extensions>
<executions>
  <execution>
   <id>bundle-manifest</id>
   <phase>process-classes</phase>
   <goals>
       <goal>manifest</goal>
   </goals>
  </execution>
</executions>
<configuration>
  <instructions>
    <Bundle-SymbolicName>custombundleactivator</Bundle-SymbolicName>
    <Private-Package>nl.rubix.custombundleactovator.*</Private-Package>
    <Bundle-Activator>nl.rubix.custombundleactovator.MyBundleActivator</Bundle-Activator>
    <Import-Package>*</Import-Package>
  </instructions>
</configuration>
</plugin>

Deploying and testing

The OSGi bundle with a custom BundleActivator requires no special treatment regarding deployment. (kinda depends on what you put in the BundleActivator class of cource but generally speaking no special action is required). Just deploy the bundle like you would deploy any other.

In this example the OSGi bundle also contains the a Camel context with one timer base route which logs a message to the fuse.log every five seconds.

When we deploy the bundle and let it run for a couple of seconds before stopping the log looks like this:

custom-bundleactivator-1

 

 
Leave a comment

Posted by on 2015-08-31 in JBoss Fuse

 

Tags: , , , , ,

ActiveMQ DLQ use OriginalDestination in Camel

Ah the DLQ, the place where messages go to die. When messages end up in the DLQ (Dead letter queue) of ActiveMQ they receive an additional message header with the queue name where the message resided originally. This message header called Original Destination, can be viewed from Hawtio.

DLQSubscriber-1

When using Apache Camel as a subscriber I noticed something strange. Normally all JMS Headers and properties are mapped to the Camel Exchange headers and properties. However, the OriginalDestination header was not.

For example when creating this dummy route for testing purposes I got the following output:


package nl.rubix.dlqsubscriber.dlqsubscriber;

import org.apache.camel.builder.RouteBuilder;

public class DLQSubscriberRouteBuilder extends RouteBuilder{

    @Override
    public void configure() throws Exception {
        from("amq:queue:ActiveMQ.DLQ").log("${headers}");
    }

}


 


[#0 - JmsConsumer[ActiveMQ.DLQ]] route1                         INFO {breadcrumbId=ID:localhost.localdomain-39943-1438348606321-13:1:1:1:1, CamelJmsDeliveryMode=2, dlqDeliveryFailureCause=java.lang.Throwable: Message Expired. Expiration:1438348926673, JMSCorrelationID=null, JMSDeliveryMode=2, JMSDestination=queue://ActiveMQ.DLQ, JMSExpiration=0, JMSMessageID=ID:localhost.localdomain-39943-1438348606321-15:1:1:1:1, JMSPriority=4, JMSRedelivered=false, JMSReplyTo=null, JMSTimestamp=1438348926663, JMSType=null, JMSXGroupID=null, JMSXUserID=null, originalExpiration=1438348926673}

I spent quite some time trying to figure out what happened when I realized the JMS message on the DLQ is not the exact same message from the original queue with some extra headers and properties. The original message seems to be wrapped inside the message placed on the DLQ. To access this message we need to work with the “raw” JMSMessage from ActiveMQ rather the processed message from Camel. This can be done by creating a processor which uses a typeconverter in Camel to get the JMS message from the Camel exchange.

In the code below I created a processor which fetches the inner message and from that message retrieves the originalDestination property (which is an instance variable on the ActiveMQMessage class (ActiveMQTextMessage is a subclass of ActiveMQMessage) and places this property as an header on the Camel exchange.


package nl.rubix.dlqsubscriber.dlqsubscriber;

import org.apache.activemq.command.ActiveMQTextMessage;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.component.jms.JmsMessage;

public class DQLMessageProcessor implements Processor{

    @Override
    public void process(Exchange exchange) throws Exception {
        JmsMessage jmsMsg = exchange.getIn().getBody(JmsMessage.class);
        ActiveMQTextMessage innerMsg = (ActiveMQTextMessage) jmsMsg.getJmsMessage();
        exchange.getIn().setHeader("OriginalDestination", innerMsg.getOriginalDestination());
    }

}


Now when observing the output log of .log(“${headers}”) we see the OriginalDestination header in the log output:


[#0 - JmsConsumer[ActiveMQ.DLQ]] route1                         INFO {breadcrumbId=ID:localhost.localdomain-39943-1438348606321-21:1:1:1:1, CamelJmsDeliveryMode=2, dlqDeliveryFailureCause=java.lang.Throwable: Message Expired. Expiration:1438350527570, JMSCorrelationID=null, JMSDeliveryMode=2, JMSDestination=queue://ActiveMQ.DLQ, JMSExpiration=0, JMSMessageID=ID:localhost.localdomain-39943-1438348606321-15:4:1:1:1, JMSPriority=4, JMSRedelivered=false, JMSReplyTo=null, JMSTimestamp=1438350527560, JMSType=null, JMSXGroupID=null, JMSXUserID=null, OriginalDestination=queue://test2, originalExpiration=1438350527570}

In the end it costs me quite some time to grab the concept of fetching the ActiveMQTextMessage from the JmsMessage, since I was under the impression the Camel ActiveMQ component does this out of the box.

Anyway I hope it helps someone.

 

 
1 Comment

Posted by on 2015-07-31 in JBoss Fuse

 

Tags: , , , , ,

JBoss Fuse 6.2 a first look at the http gateway

Tech preview in 6.1 the http gateway for Fuse fabric is in full support in JBoss Fuse 6.2. The http gateway (and mq gateway for ActiveMQ) offers a gateway for clients not residing inside the fabric to services within the fabric. When not using this kind of functionality services have to be running on particular IP addresses and ports for clients to connect to them. Even when using a loadbalancer, the loadbalancer has to know on what IP and port combination the services are running. The whole concept of fabric and containers in general is that they become IP agnostic, it does not really matter where and how many services/containers are running as long as they can handle the load. This IP agnosticism introduces complexity for clients who need to connect to these services. The http gateway offers a solution for this complexity by using autodiscovery based on the Zookeeper registry of Fuse fabric and exposes itself for external clients on one static port. So now only the http gateway needs to be deployed on a static IP and port, while the services can be deployed on any container inside the fabric.

Prior to the 6.2 release this autodiscovery mechanism needed to be built automatically and a dumbed down version of the gateway could be build using Camel. I have blogged about creating both a service and a client (acting as a dumbed down gateway, I called it proxy in my post) here:

https://pgaemers.wordpress.com/2015/02/26/fabric-load-balancer-for-cxf-part-1/
https://pgaemers.wordpress.com/2015/03/16/fabric-load-balancer-for-cxf-part-2/

 

Now that the http gateway is in full support let’s take a look at it.

Deploying a service

To hit the ground running I’ve used the Rest Quickstart example provided in JBoss Fuse. I just simply create a new fabric container with the quickstarts/cxf/rest profile:

http-gateway-1

After the container is started it provides a Rest service with four verbs, for the purpose demoing the http gateway we will be using only the get operation since it is the simplest to invoke.http-gateway-7a

In Hawtio when navigating to Services -> APIs we can see the API provided by the REST quickstart is registered inside Fuse. We can browse the endpoint or retrieve the WADL.

http-gateway-2When invoking the get operation from a browser we can fetch a response.

http-gateway-3

When we connect to the container and inspect the log file we can see the following entries:

We can see the invoked endpoint is: http://10.0.2.15:8182/cxf/crm/customerservice/customers/123

Deploying the http gateway

Deploying the standard out of the box http gateway couldn’t be simpler. Just spin up a new container and add the Gateway/http profile to it.

http-gateway-5

By default the http gateway is exposed on port 9000 and since we do not have any custom mapping rules implemented the context path of the service is used by the gateway as is. This means we can use the same URI as the previous request except for the port which is now 9000. So when invoking this endpoint we get the same response we previously got: http://10.0.2.15:9000/cxf/crm/customerservice/customers/123/

http-gateway-6

When looking at the log file of the container hosting our service we can now see the following entries:

http-gateway-4

We can see the request came from port 9000

Replicating the service

But what happens when we create another instance of our rest quickstart service by creating another container with the same profile. Or replicating our container.

Since the rest quickstarts has a relative path configured JBoss Fuse Fabric allocates a port for when provisioning a container with the rest quickstart. In our first container this was 8182. To see the endpoint of our new container we can again go to the Services -> APIs tab in Hawtio and take look:

http-gateway-8

Here we see the latest instance of the service is bound to port 8184, and indeed when we invoke this endpoint we retrieve the expected response:

http-gateway-9When using the gateway endpoint (port 9000) to invoke the service we can observe the logs of both containers to see gateway requests are fed to both endpoints:

http-gateway-10

http-gateway-11

This means clients wanting to invoke this service can use the gateway endpoint, in this example invoke the 9000 port and requests get automatically redirected to the service containers. Dynamically reallocating the services to new IP addresses and/or ports does not effect this as long as the service is inside the fabric and the endpoint is registered inside Zookeeper!

 

Configuring the http gateway

Until now we have been running the http gateway in the standard configuration. This means the rules for redirecting are taking the contextpath of the service. That’s why we only had to change the port number to direct our browser to the gateway rather than one of the service directly. It is however possible to change the configuration of the gateway profile and have different mapping rules.

To change the configuration of the http gateway we have to modify the properties stored in the gateway/http profile. Make sure you modify the properties in the profile and not in the child container running the profile since this can cause synchronization problems inside the fabric. When changing properties always do it in the profile/configuration registry.

This can be done by utilizing the GIT api of the Fabric configuration registry directly or by using Hawtio. For this demo we are going to use Hawtio.

http-gateway-12

When using Hawtio to browse to the http gateway profile we can see the four property files containing the mapping rules:

  • fabric8.gateway.http.mapping-apis.properties
  • fabric8.gateway.http.mapping-git.properties
  • fabric8.gateway.http.mapping-servlets.properties
  • fabric8.gateway.http.mapping-webapps.properties

The mapping rules, and property files, are devided in four different categories. In the documentation it is explained what those categories are: https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.2/html/Fabric_Guide/Gateway-Config.html

Our rest quickstart falls under the mapping-apis category so we will have to modify the “io.fabric8.gateway.http.mapping-apis.properties” file. This can be done straight from Hawtio.

Simply click that particular property file and the read-only mode opens.

http-gateway-13

Here we see the path in the Zookeeper registry where the endpoints of all APIs are located. To edit the file simply click edit.

As mentioned before the default mapping rule maps the zookeeperPath to the contextpath. But we can insert our own paths.

To modify the mapping add the following property to the property file:

uriTemplate: /mycoolpath{contextPath}/

The complete file looks like this:

#
#  Copyright 2005-2014 Red Hat, Inc.
#
#  Red Hat licenses this file to you under the Apache License, version
#  2.0 (the "License"); you may not use this file except in compliance
#  with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
#  Unless required by applicable law or agreed to in writing, software
#  distributed under the License is distributed on an "AS IS" BASIS,
#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
#  implied.  See the License for the specific language governing
#  permissions and limitations under the License.
#

zooKeeperPath = /fabric/registry/clusters/apis
uriTemplate: /mycoolpath{contextPath}/

Now when we go to this url: http://localhost:9000/mycoolpath/cxf/crm/customerservice/customers/123/ we get the response from the service.

http-gateway-14

In the example above we just changed the uriTemplate property. For more finegrained URL rules we can add extra rules also modifying the zooKeeperPath property. For example to split REST and SOAP APIs.

The http gateway is a very handy tool to solve the problem of having to know what service is running on what container binding to what port. Since it uses the Zookeeper registry of the fabric it can enable making the services container agnostic. It should also help tremendously when securing the fabric. The gateway can drastically reduce the amount of ports that need to be opened for the outside world and can make configuring external proxies and load balancers a lot easier.

 
Leave a comment

Posted by on 2015-07-18 in JBoss Fuse

 

Tags: , , , , ,

JBoss Fuse 6.2 – a first look at the data transformation mapper

JBoss Fuse 6.2 is just release and one of the new exiting features is the new, graphical, data transformation component. And although still in tech preview phase I got curious and started to play with it. Now there are already some good tutorials available. For example this one: https://vimeo.com/131250890

After tinkering around a bit with the new feature I found some peculiarities which I will share here.

Project setup

I decided to go with a similar setup as in the tutorial described above. Transforming a xml document into a JSON document.

The xsd for the xml document I used looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Person">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="PersonDetails">
          <xs:complexType>
            <xs:sequence>
              <xs:element type="xs:string" name="firstName"/>
              <xs:element type="xs:string" name="lastName"/>
              <xs:element type="xs:integer" name="age"/>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element name="Addresses">
          <xs:complexType>
            <xs:sequence>
            <xs:element name="Address" minOccurs="0" maxOccurs="unbounded">
                <xs:complexType>
                    <xs:sequence>
                        <xs:element type="xs:string" name="street"/>
                        <xs:element type="xs:string" name="city"/>
                        <xs:element type="xs:string" name="country"/>
                    </xs:sequence>
                </xs:complexType>
            </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

The JSON sample doc looks like this:

{"name":"dummyName",
"sirName":"dummySirName",
"age":"32",
"addresses":[
{"streetname":"dummyStreet","cityname":"dummyCity","countryname":"dummyCountry"},
{"streetname":"dummyStreet2","cityname":"dummyCity2","countryname":"dummyCountry2"}
]}

Since we are concentrating on the data transformation the rest of the project setup is not really all that important. I initially created a pass-through route with a direct and a mock endpoint:

datatransformation-1When creating a new data transformation drag and drop the “Data Transformation” component from the palette to the canvas.

datatransformation-2This will bring up the wizard for setting up the data transformation.

datatransformation-3The “Transformation ID” field is a unique identifier for the transformation, similar to the id field in a bean defined in the OSGi Blueprint configuration. The “Source Type” and “Target Type” drop downs allow you to select the source and target specifications.

datatransformation-4The choices in the drop down are Java, XML, JSON and other. Other basically means you have to generate/create your own java type representation of the message format. For example using Bindy or BeanIO for CSV messages. Using the XML and JSON options will generate the java object for you on the fly, which is very handy.

When selecting XML as the source and JSON as the Target type and click next the next screen is the xml type window.

datatransformation-5Here we can select our XML schema as source file and select Person as root element. A preview is displayed in the “XML Structure Preview” window.

datatransformation-6Clicking “next” we move on to the JSON type window.

datatransformation-7

When clicking “Finish” the graphical Data Transformation screen is displayed and we can begin mapping the fields.

datatransformation-8

To map the fields simply drag the source field to the required target field.

Note, since we have repeating groups in both the xml and JSON messages we also need to drag and drop address to addresses. The complete mapping looks like this:

datatransformation-9

Now we have to wire the components in the route together to finish things up.

datatransformation-10

Peaking under the covers:

When switching to the source mode of the Blueprint configuration we can see what the wizard has setup:


<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
       http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">

  <camelContext xmlns="http://camel.apache.org/schema/blueprint">
  <endpoint uri="dozer:myfirstdatatransform?sourceModel=persons.Person&amp;targetModel=personsjson.Personsjson&amp;marshalId=transform-json&amp;unmarshalId=persons&amp;mappingFile=transformation.xml" id="myfirstdatatransform"/>
  <dataFormats>
    <jaxb contextPath="persons" id="persons"/>
    <json library="Jackson" id="transform-json"/>
  </dataFormats>
  <route>
    <from uri="direct:start"/>
    <to ref="myfirstdatatransform"/>
    <to uri="mock:output"/>
  </route>
</camelContext>

</blueprint>

As we can see an endpoint has been created and configured with the Dozer component. Also two data formats have been created representing the xml schema and JSON documents respectively.

When looking at the project explorer we can also see the generated Java classes:

datatransformation-11

 

Testing the transformation

Now that our transformation is finished we need to test it. Luckilly Fuse provides the generation of a unit test for data transformations:

datatransformation-12In the wizard simply point to the Blueprint containing the data transformation and the data transformation file and hit finish:

datatransformation-13This generates a Camel Junit test, containing a stub Camel route, a producertemplate and a helper method for reading files (for feeding the producertemplate). The generated test class looks like this:


package nl.rubix.datatransformation.datatransformationtest;

import java.io.FileInputStream;

import org.apache.camel.EndpointInject;
import org.apache.camel.Produce;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.blueprint.CamelBlueprintTestSupport;
import org.junit.Test;

public class TransformationTest extends CamelBlueprintTestSupport {
    
    @EndpointInject(uri = "mock:myfirstdatatransform-test-output")
    private MockEndpoint resultEndpoint;
    
    @Produce(uri = "direct:myfirstdatatransform-test-input")
    private ProducerTemplate startEndpoint;
    
    @Test
    public void transform() throws Exception {
        
    }
    
    @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {
            public void configure() throws Exception {
                from("direct:myfirstdatatransform-test-input")
                    .log("Before transformation:\n ${body}")
                    .to("ref:myfirstdatatransform")
                    .log("After transformation:\n ${body}")
                    .to("mock:myfirstdatatransform-test-output");
            }
        };
    }
    
    @Override
    protected String getBlueprintDescriptor() {
        return "OSGI-INF/blueprint/blueprint.xml";
    }
    
    private String readFile(String filePath) throws Exception {
        String content;
        FileInputStream fis = new FileInputStream(filePath);
        try {
             content = createCamelContext().getTypeConverter().convertTo(String.class, fis);
        } finally {
            fis.close();
        }
        return content;
    }
}

To complete our test just fill in our test method and we’re good to go. For this example we are just going to feed the producertemplate with a sample document and check the log. Normally you would create a more detailed unit test containing assertions and what not.

Our filled in test method looks like this:


@Test
public void transform() throws Exception {
 startEndpoint.sendBody("direct:myfirstdatatransform-test-input", readFile("src/test/resources/dummyInput.xml"));
}

Now run it as a Junit test and inspect the log.

When inspecting the log we can see our transformation working and producing the following JSON document:

{"name":"testFirstName","sirName":"testLastName","age":"32","addresses":[{"streetname":"teststreet","cityname":"testcity","countryname":"testcountry"},{"streetname":"teststreet2","cityname":"testcity2","countryname":"testcountry2"}]}

A simple use case and a gotcha

Now above we created one of the simplest transformations imaginable. A common use case and still pretty straightforward is performing some filter action inside a loop. In this example we want to filter out some addresses in our transformation. As an example we only want to transform the address with “testcountry2” and ignore all others. A real life scenario would for example filter all billing addresses from postal addresses.

The data transformation offers next to the basic field mapping we’ve looked at previously also a couple of other options.

When selecting the mapping we want add the filter (address -> addresses) we can click to get the extra options:

datatransformation-14

The “Set field” option is just what we already performed to select a target and source field for the transformation. The “Set variable” allows us to use a variable we can define on the variable tab on the source pane. The “Set expression” allows us to leverage on of the Camel supported expression languages to perform some data manipulation. And finally the “Add custom function” allows us to create a Java helper method so some custom transformation logic can be implemented.

Since using XML as source for the data transformation, I initially tried to create an xpath expression performing the filtering. After all filtering in xpath is quite easy to do. However here I experienced some behaviour I did not expect.

In the menu above select “Set expression”, a new window pops up:

I selected xpath and entered the following expression:

/Person/Addresses/Address[country = “testcountry2”]

However, when observing the output log I encountered the following output:

{"name":"testFirstName","sirName":"testLastName","age":"32","addresses":["<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n<Address>\n           <street>teststreet2</street>\n           <city>testcity2</city>\n           <country>testcountry2</country>\n       </Address>"]}

Basically, two things have happened, first the filtering is performed correctly, but the output of the filter is “paste” into the target field. This is also mentioned in the tutorial I linked to above, but this does mean this simple use case is not supported using expressions. This also seems a bit unintuitive. To implement the filter use case I ended up creating a custom function.

After removing the xpath, by drag and drop the address field again. In the menu presented in the last screenshot select “Add custom function”. The following wizard pops up:

datatransformation-15

Click the “C” button to create a new class. In the screen that follows, select a Java package, enter a Class name and select the Return type and Parameter type.

datatransformation-16

The following class is generated:


package nl.rubix.datatransformation.datatransformationtest;

public class FilterTest {

    public java.util.List< ? > map(java.util.List< ? > input) {
        return null;
    }

}

Luckily we can use the new Java 8 collection types and lambda expressions which eases the implementation of our filtering. The finished FilterTest class looks like this:


package nl.rubix.datatransformation.datatransformationtest;

import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;

import persons.Person.Addresses.Address;

public class FilterTest {

    public ArrayList<personsjson.Address> map(ArrayList<Address> input) {
        List<personsjson.Address> filteredAddress = input.stream().filter(c -> c.getCountry().equals("testcountry2")).map(Address -> createAddress(Address)).collect(Collectors.toList());
        
        return (ArrayList<personsjson.Address>) filteredAddress;
    }
    
    private personsjson.Address createAddress(Address personAddress){
        personsjson.Address address = new personsjson.Address();
        address.setCityname(personAddress.getCity());
        address.setCountryname(personAddress.getCountry());
        address.setCountryname(personAddress.getCountry());
        
        return address;
    }

}

Now when we run the unit test and observe the logging we can see the correctly formatted JSON.

{"name":"testFirstName","sirName":"testLastName","age":"32","addresses":[{"cityname":"testcity2","countryname":"testcountry2"}]}

Although the data transformation functionality is very nice, I am a bit disappointed that for such a simple and common use case as filtering a custom Java class has to be created. This will probably mean for most data transformation we still cannot eliminate custom code. Something a graphical data transformation tool should aim at. But in all fairness the data transformation is still in tech-preview phase so who knows how it will look like and perform when it is officially supported.

 
Leave a comment

Posted by on 2015-07-05 in JBoss Fuse

 

Tags: , , ,