Stopping a fabric container after ‘container has not been created by fabric’ error message

When creating a new fabric sometimes there are still containers running in the background of the old fabric. These can cause conflicts with the ‘new’ containers. Especially when using the same ZooKeeper credentials as the old fabric in the new one.

Note this posts will only explain how to stop and delete a container after the ‘container has not been created by fabric’ error message. It may very well be that other configurations and background processes are running of the old fabric what may cause unexpected behavior. If you have the possibility to start your fabric from scratch (for intance a developer machine) it may be better to restart fuse with ./fuse -clean and create a new fabric with the command fabric:create –clean

for more information about the available commands see the Fuse documentation:

https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.1/html/Console_Reference/files/ConsoleRefIntro.html

One of the problems is that when trying to start, stop or delete a container you get the error message ‘container <<your container>> has not been created by fabric’.

container-stop-error-1

Also trying to stop/start/delete the container from the Fuse karaf console will result in the same message.

JBossFuse:karaf@root> fabric:container-list
[id] [version] [connected] [profiles] [provision status]
root* 1.0 true fabric, fabric-ensemble-0000-1 success
  demoContainer 1.0 true jboss-fuse-full, GettingStarted success
JBossFuse:karaf@root> fabric:container-stop demoContainer
Error executing command: Container demoContainer has not been created using Fabric

So how can we stop this container?

To stop this container we need to take two steps;

1) install the ‘zookeeper-commands’ feature in the root container

2) using zookeeper commands to find the PID of the container so we can kill it through the os.

1) installing the ‘zookeeper-commands’ feature in the root container

In fabric8 features are also controlled by fabric8, for this reason the good old ‘feature:install’ command will not work.

I resulted in installing the zookeeper-commands feature through the Hawtio console.

In the Hawtio console go to ‘Wiki’ → edit fabric features → search for zookeeper and select zookeeper-commands → click the ‘Add’ buttom and finally click ‘save’

In Hawtio console go to wiki and select the fabric profile.

container-stop-error-2

Then select the edit features button at the bottom of the features list:

container-stop-error-3

Search for zookeeper, select zookeeper-commands → click ‘Add’ → click ‘Save changes’

container-stop-error-42) Stopping the container

After installing the zookeeper-commands feature we can use the zookeeper commands in the Fuse/karaf shell:

finding the PID of the container we want to stop execute the following command:

JBossFuse:karaf@root> zk:list -r -d|grep demoContainer|grep pid
/fabric/registry/containers/status/demoContainer/pid = 7177

Now in a ‘regular’ shell we can kill the process (and thus the container) with this PID.

[jboss@localhost ]$ ps -ef|grep 7177
jboss     7177     1  2 12:30 pts/0    00:03:02 java -server -Dcom.sun.management.jmxremote -Dzookeeper.url=10.0.2.15:2181 -Dzookeeper.password.encode=true -Dzookeeper.password=ZKENC=YWRtaW4= -Xmx512m -XX:+UnlockDiagnosticVMOptions -XX:+UnsyncloadClass -Dbind.address=0.0.0.0 -Dio.fabric8.datastore.component.name=io.fabric8.datastore -Dio.fabric8.datastore.type=caching-git -Dio.fabric8.datastore.felix.fileinstall.filename=file:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/etc/io.fabric8.datastore.cfg -Dio.fabric8.datastore.service.pid=io.fabric8.datastore -Dio.fabric8.datastore.gitPullPeriod=1000 -Djava.util.logging.config.file=/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/instances/demoContainer/etc/java.util.logging.properties -Djava.endorsed.dirs=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.7.0.fc20.x86_64/jre/jre/lib/endorsed:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.7.0.fc20.x86_64/jre/lib/endorsed:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/endorsed -Djava.ext.dirs=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.7.0.fc20.x86_64/jre/jre/lib/ext:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.7.0.fc20.x86_64/jre/lib/ext:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/ext -Dkaraf.home=/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379 -Dkaraf.base=/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/instances/demoContainer -Dkaraf.data=/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/instances/demoContainer/data -Dkaraf.etc=/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/instances/demoContainer/etc -Dkaraf.startLocalConsole=false -Dkaraf.startRemoteShell=true -classpath /home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/org.apache.servicemix.specs.activator-2.3.0.redhat-610379.jar:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/karaf.jar:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/karaf-jaas-boot.jar:/home/jboss/Apps/jboss-fuse-6.1.0.redhat-379/lib/esb-version.jar org.apache.karaf.main.Main
[jboss@localhost ]$ kill -9 7177

Now the container is stopped, but it still exists. And we still can’t start it and stop it through the regular process. So in the next step I’ll explain how to delete the container.

deleting the container

To remove the container from the fabric we need to remove all the zookeeper entries. To do this first we first need to know what entries we need te remove.

We can use the zk:list command again for retrieving the zookeeper entries.

JBossFuse:karaf@root> zk:list -r|grep demo
/fabric/configs/containers/demoContainer
/fabric/configs/versions/1.0/containers/demoContainer
/fabric/registry/containers/config/demoContainer
/fabric/registry/containers/config/demoContainer/bindaddress
/fabric/registry/containers/config/demoContainer/maximumport
/fabric/registry/containers/config/demoContainer/localhostname
/fabric/registry/containers/config/demoContainer/localip
/fabric/registry/containers/config/demoContainer/publicip
/fabric/registry/containers/config/demoContainer/parent
/fabric/registry/containers/config/demoContainer/resolver
/fabric/registry/containers/config/demoContainer/minimumport
/fabric/registry/containers/config/demoContainer/manualip
/fabric/registry/containers/config/demoContainer/ip
/fabric/registry/ports/containers/demoContainer
/fabric/registry/ports/containers/demoContainer/org.apache.karaf.shell
/fabric/registry/ports/containers/demoContainer/org.apache.karaf.shell/sshPort
/fabric/registry/ports/containers/demoContainer/org.apache.karaf.management
/fabric/registry/ports/containers/demoContainer/org.apache.karaf.management/rmiServerPort
/fabric/registry/ports/containers/demoContainer/org.apache.karaf.management/rmiRegistryPort
/fabric/registry/ports/containers/demoContainer/org.ops4j.pax.web
/fabric/registry/ports/containers/demoContainer/org.ops4j.pax.web/org.osgi.service.http.port

Now we know all the entries of the container we want te delete we can delete the entries using the zk:delete command. Make sure you use the -r parameter to recursively remove alle entries.

JBossFuse:karaf@root> zk:delete -r /fabric/configs/containers/demoContainer
JBossFuse:karaf@root> zk:delete -r /fabric/configs/versions/1.0/containers/demoContainer
JBossFuse:karaf@root> zk:delete -r /fabric/registry/containers/config/demoContainer
JBossFuse:karaf@root> zk:delete -r /fabric/registry/ports/containers/demoContainer

Now when we execute the fabric:container-lists command the demoContainer is no longer in the list.

JBossFuse:karaf@root> fabric:container-list
[id]                           [version] [connected] [profiles]                                         [provision status]
root*                          1.0       true        fabric, fabric-ensemble-0000-1                     success