Tuesday, March 13, 2012

solved Oracle NoSQL java.net.NoRouteToHostException

When you are installing Oracle NoSQL on an Oracle linux machine you can follow the Oracle guide which will run you through the simple installation process. There are however some things to keep into consideration. One of the steps is to do a check if you key-value store database is up and running. When you use a default Oracle Linux installation you will most likely succeed when you do a ping to the same host that you are working on. The issue however is starting to occur when you try to do a ping to another machine.

The following situation:
nosql0.exampledomain.com --
nosql1.exampledomain.com --

When you are on nosql0 and execute the following below command you will have a positive result.

java -jar ./lib/kvstore-1.2.123.jar ping -port 5000 -host nosql0.exampledomain.com

If you are on nosql1 and you execute the below command you will also get a positive result.
java -jar ./lib/kvstore-1.2.123.jar ping -port 5000 -host nosql1.exampledomain.com 

However, if you are on nosql0 and try to ping the nosql1 with the below command you will get an error.
java -jar ./lib/kvstore-1.2.123.jar ping -port 5000 -host nosql1.exampledomain.com  

if you use a ping command you can ping the other server (if not you have another network issue) and you can setup a SSH session however the error message will state that you do not have a route to the mentioned host. The error message will look something like the one below:

[nosql@nosql0 kv-1.2.123]$ java -jar ./lib/kvstore-1.2.123.jar ping -port 5000 -host nosql1.terminalcultexample.org
Exception in thread "main" java.rmi.ConnectIOException: Exception creating connection to: nosql1.terminalcultexample.org; nested exception is:
        java.net.NoRouteToHostException: No route to host
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:632)
        at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
        at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
        at sun.rmi.registry.RegistryImpl_Stub.list(Unknown Source)
        at oracle.kv.util.Ping.getTopology(Ping.java:332)
        at oracle.kv.util.Ping.main(Ping.java:104)
        at oracle.kv.impl.util.KVStoreMain$8.run(KVStoreMain.java:218)
        at oracle.kv.impl.util.KVStoreMain.main(KVStoreMain.java:319)
Caused by: java.net.NoRouteToHostException: No route to host
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
        at java.net.Socket.connect(Socket.java:546)
        at java.net.Socket.connect(Socket.java:495)
        at java.net.Socket.(Socket.java:392)
        at java.net.Socket.(Socket.java:206)
        at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
        at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
        ... 8 more
[nosql@nosql0 kv-1.2.123]$

As it turns out by default Oracle Linux will have iptables configured and this will block your connection to port 5000. You can check you iptables settings by issuing the following command: iptables -L -n

This will give you something like the below:
[root@nosql1 init.d]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --             state RELATED,ESTABLISHED
ACCEPT     icmp --  
ACCEPT     all  --  
ACCEPT     tcp  --             state NEW tcp dpt:22
REJECT     all  --             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
REJECT     all  --             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
[root@nosql1 init.d]#

What you need to do is configure iptables to allow network traffic to port 5000 or disable iptables. Disabling iptables is never a smart move however you can opt for it in some cases. 

Monday, March 12, 2012

Duqu trojan payback for decommissioning old IBM systems

Our society is more and more depending on computers. Financial transactions are done mosntly via computer transactions, industries are depending on it and armed forces are mostly blind and without information if the IT systems supporting them stop working. We do have to worry about solar storms knocking out most of our modern day communication channels and systems however somewhat closer to earth we also have some things to worry about.

One of the threads we have to worry about and which is coming from our own planet is the fact that criminals and not only criminals but also the military and secret service of countries are working on very advanced technology to knock out systems, break into them or cause other distortion and theft. We are not talking about hackers, as I still do have a mindset in which I see hackers as the good guys who do play intelligent games that sometimes are just reaching over the line of the officially legal. However in my opinion hackers are still the good guys.

It is the people who turn to the true dark side and do sell their craft to wealthy criminal organizations. Some very gifted developers and computer scientists go for the big bugs and do not care about what they develop and with what intention it will be developed.

The second group is the group of computer scientists who do sell their craft to governments in the form of working for an army of secret service. This group is somewhat more discussable on the fact if they go for the monetary pleasure or not. It is without any doubt that governments are willing to pay top dollar for gifted developers and computer scientists however we have to keep in mind that one man his terrorist is the other his freedom fighter.  We can state we do agree or disagree with some of the thoughts of other governments however I do think that this is less dollar driven.

That a lot of money is paid to developers to develop virus code and tools to cause mayhem is shown again by the people from the Kaspersky security lab. The Kaspersky lab is currently trying to find out how the new Duqu Trojan is developed and how it is working. What they have found up until now is what it is doing and how it is communication. The scary part however of this Trojan is that it is developed in a language that we do not know. It is tested to see if it is developed in C++, Objective C, Java, Python, Ada, Lua, or any other languages however all tests are currently negative.

Developing a new programming language is a very long and costly process and will need very experienced developers. Developing a new programming language needs a wealthy backing in the form of a government or very wealthy criminal syndicate. However there is another option, the option that it is not a new language however a very old language.  Some people claim that it might be the result of an old IBM compiler used in OS400 SYS38 and the oldest sys36 systems.

That code looks familia:r
The code your referring to .. the unknown c++ looks like the older IBM compilers found in OS400 SYS38 and the oldest sys36.

The C++ code was used to write the tcp/ip stack for the operating system and all of the communications. The protocols used were the following x.21(async) all modes, Sync SDLC, x.25 Vbiss5 10 15 and 25. CICS. RSR232. This was a very small and powerful communications framework. The IBM system 36 had only 300MB hard drive and one megabyte of memory,the operating system came on diskettes.

This would be very useful in this virus. It can track and monitor all types of communications. It can connect to everything and anything.

Some parts of the current Duqu Framework are “simple” C++ code however some parts are written in the unknown code which might be related to the above quote from As400tech (looking at his knowledge and his name would be a very experienced AS400 developer). If this turns out to be true it could mean that it could be that the developer of this part of the Trojan is an experienced AS400 developer. As we see that companies are decommissioning AS400 systems daily and that they leave an entire community of AS400 developers behind without a proper job this could mean a large group of people comes to the market that are potentially very interesting to governments, secret services and criminal syndicates. Whoever stated that AS400 developers where out of the market was apparently wrong.

However, it is only a thought of one person and not necessarily correct. Some people do think it is coded low level assembly code. This would mean that someone has taken the task upon himself to write all the assembly code himself instead of using a compiler to build it into machine language. However the person is who created the Duqu Trojan (and the Stuka) it must have been a very good programmer or a team of good programmers ( in my humble opinion).

You can condemn the writing of such a code from an ethic and moral point of view, you can agree with it, in any case whatever your point of view on this is you have to admire the craftsmanship of the developer.

Friday, March 09, 2012

When to use Hadoop

Hadoop is one of the big players in the big-data and can be seen as one of the main engines running the big-data machine. We however still do not have a clear picture on what is big-data. we do have some definitions on when we call a lot of data big data however giving it a number has not been done up until now and will most likely never been done. I already zoomed in into this definition question in the "Map reduce into relation of Big Data and Oracle" post on this blog. A number of key components state if data is big-data, to name them; volume of the data, the velocity in which the data grows, the variety of sources which add to the volume of the data and the value it can "potentially" hold. These factors can help you decide when data is big data.

Then we have the question on when data (even big-data) can still be handled in a standard relational database and can still be handled by a "standard" approach. There are some guidelines that can help you. Please do note this is a comparison primarily  for handling data in a relational database or in Hadoop. This is not for storing data.

RDBMS Hadoop / MapReduce
Data Size Gigabytes Petabytes
Access Interactive and batch Batch
Structure Fixed Schema Unstructured schema
Language SQL Procedural (Java, C++, Ruby, etc.)
Integrity High Low
Scaling nonlinear linear
Updates Read and Write Write ones, read many times
Latency Low High

By taking this into consideration when you are struggling with the question if you need to use a MapReduce approach or a RDBMS approach it might be a little more easy to make your decision.