Perl use an array variable

Perl, as many other languages has a array variable type. Accoording to Wikipedia an array is the following in computer science: In computer science, an array type is a data type that is meant to describe a collection of elements (values or variables), each selected by one or more indices (identifying keys) that can be computed at run time by the program. Such a collection is usually called an array variable, array value, or simply array. The use of an array is a very effective way of storing variabels in something that is most comparable to a list of things.

In the below example we stored some of the names of a phonetic alphabet in the variable phonetic:

my @phonetic = ( "Alpha", "Bravo", "Charlie", "Delta" );

Now if we want to do something with it we can call the variable @phonetic however this would give you the entire collection of all values in the array. For example using the print command:

my @phonetic = ( "Alpha", "Bravo", "Charlie", "Delta" );

print @phonetic;

this would give you a result you most likely do not want, namely:

AlphaBravoCharlieDelta

As you can see this is printed without any space between it or a newline. Simply using all the values from the array in the print command at once. In many cases you would like to take all the values of the array one by one. In some cases, for example adding up all the numerical values in the array, you might want to use it in this way however in most cases you would like to loop value per value.

In the example below we loop the array and do a print for every value in the array.

my @phonetic = ( "Alpha", "Bravo", "Charlie", "Delta" );

foreach (@phonetic) {
 print $_ . "\n";
}

When running this example you will see that we do not get the result all in one line as was with direct print on the array. Now we will have a result as shown below;

Alpha
Bravo
Charlie
Delta

In some cases you would like to do an action on a certain value. Every value in an array has an index number (starting at 0). So lets say we want to print the value Charlie we have to call the array value with the index number 2. The below example will print the value "Charlie"

my @phonetic = ( "Alpha", "Bravo", "Charlie", "Delta" );

print @phonetic[2];

A lot of people do read information online and use online resources when figuring out how to code solutions. Online a lot of great resources can be found to support developers. One thing a lot of people do not think about however is that there is, in my opinion, some moral duty to also share back. People who do use online resources for free and learn from it and even make profit with the gathered knowledge should also share back. When you do share information it is good to remember also how you as a consumer of information would like to read the information. When it comes to code for example it is nice to see a good formated piece of code.

When placing code online it is good to do this in a way that it is formated in a nice way so that reading the code is more easy. To help you with this task when blogging you can make use of the Google Code Prettify project. The Google Code Prettify project brings you support for syntax highlighting of code snippets in a web page. For example the code part below is done by making use of this.

class Voila {
public:
  // Voila
  static const string VOILA = "Voila";

  // will not interfere with embedded tags.
}

The Google Code Prettify project has a number of options for syntax highlighting for specific languages. You can find a list of them at the google code page. here you can also find ways on how to implement the Google Code Prettify solution on your blogger page and or your customer website.

Oracle database security blueprint against network attacks

Databases play a vital role in many current enterprise systems. They are commonly used to store vital, critical and confidential data about customers, finance, logistics and other operations within the company. Due to the central role a lot of database play in an overall architectural landscape of a company it can be expected that companies do take all measures to ensure the security of a database. Security can be seen from many different angles. For example availability is a security point which is often not considered to be part of security. When people talk about security in general they think about how to protect unwanted and unauthorised people from accessing a system or data.

When thinking about how to protect a system, a database in this case from being accessed by people who are not intended to a lot of people do think in the following order about security. User accounts, networking, applications, operating systems and then the rest. All are evenly important however thinking about security is something that needs to be taken very carefully. For example the network security is not simply placing a firewall between the database and the application server or directly to the rest of the world.

Below is a start of a blueprint which might help you to start your own database security blueprint. In this case we have taken a situation in which the database is used in combination with an application server which is connected to the public internet. A couple of things to remind, we only take network security as a topic in this blogpost and we do only think about security in a way to prevent users from attacking the database via the network to gain access. Meaning this rules out DDOS kind of attacks and this rules out any attacks on the application server (directly).

In the image below you can see the implementation of the blueprint for a database in a more then average secured landscape. This however is not yet considered a full secure architecture however is providing you security against a large number of the general attacks towards the database that might be undertaken on a web-facing application.

It is common practice for most companies to place web-facing application servers in a DMZ for security reasons. What is not common practice is that both firewalls should be of a different make and model. Reason for this is that if an attacker would be able to compromise the first firewall it would be very simple to hack the second DMZ-inside firewall when this was of the same make and model.

Next to this you can notice that the application server is attached to two different (V)lan's. Reason for this is that on the user (V)LAN you most likely only want to have one singel port open which is exactly the same port you will allow to be accessed from the outside world. Due to this setup it is important that you have at least two different NIC's. One NIC attached to the User VLAN and one attached to the application VLAN. On the application VLAN you can have more ports open then you will have on the  user VLAN. As you can see all servers in the above shown blueprint design are hardend by themselves by making also use from a local firewall. This means that even though you have firewalls available on network level you also have on every server a local firewall as an extra layer of security. On Linux servers you would use ipTables for this.

In your application design and your database schema and user design you have to already have to made sure most common security features are available. For example by making use of deep application boundary validation in your code which I already discussed in a previous blogpost.

On a more database security topic, in the blueprint design shown above you can see that the application server is not connected to the database server directly. Instead it is connected to the a database firewall server which sits in its own Oracle database firewall DMZ. The reason the Oracle database firewall is placed here is that all the other firewalls, also the Linux internal firewalls, are only there to protect against unauthorised network routing. Those firewalls simply state if a connection between 2 systems on a specified port is allowed. The Oracle database firewall is adding to this that it is checking the actual SQL statements that are executed. The Oracle database firewall is protection you against a potential attack via, for example, SQL injection. I have been discussing the Oracle database firewall in more detail in a previous blogpost.

The Oracle database firewall will be the point to which your application server will connect to like it is a normal database. The Oracle database firewall will check the statements it receive for a specific database against a whitelist of statements and if approved act as a "proxy" towards the database. This is shown in the below image from Oracle.

Now we have in place firewalls on the hosts protecting the hosts with IPtables. we also have firewalls in between the network segments and have separated the different network segments. We have also deployed a Oracle database firewall to ensure that not only the network traffic is controlled we have also ensured that the statements that are send to the database are valid and do not contain any statements that could be used to exploit the database.

As a last line of defence we use a technique that is less known even by most Oracle DBA's. We limit the hosts that can connect to the database on the database itself. In case someone is able to circumvent all firewalls and bypass the Oracle database firewall we have an option to state in the database instance itself which hosts can connect. A sort of whitelist of hosts, a sidenode to this is that you will have to add for example your application servers on this list and the Oracle database firewall. If someone gains access to a shell on those servers and starts a SQL session from this server it is considered valid. However it will hold back all the SQL sessions from IP's that are not in the whitelist.

To enable this valid node checking function you have to add some information to your $TNS_ADMIN/sqlnet.ora configuration. You have to change (add) the following to the file:

tcp.validnode_checking = YES
tcp.invited_nodes = ( X.X.X.X, hostname, ... )

Do note that if you do not add the IP's or hostnames of the machines your DBA is using they will also be unable to connect to the system. Adding the tcp.validnode_checking option to YES is in the security best practices of Oracle and should (in my opinion) be done always unless you have a very valid point to not do this.

A good thing to note is that if you use tcp.validnode_checking in an Oracle eBS setup this is supported by the autoconfig functionality. AutoConfig supports automated configuration of this setting. If the profile option “SQLNet Access” (FND_SQLNET_ACCESS) is set to “ALLOW_RESTRICTED” at the Site level when AutoConfig is run on the database server, AutoConfig will add IP restrictions to sqlnet.ora. The list of host will be all those from the FND_NODES table that are registered as an EBS node.

For more information, refer to MOS Note 387859.1: Using AutoConfig to Manage System Configurations with Oracle Applications Release 12 - or the Oracle Applications Concepts manual.

Cloud computing risk of data silo

Companies are more and more moving to the cloud. Cloud applications in the form of SaaS are getting more common in enterprises, hosting in the cloud in the form of DBaaS, PaaS or IaaS are getting more common by the day. I the"early" days of cloud computing companies who adopted where more small and medium businesses and startup companies. Now we see a move from enterprises towards the cloud. Large vendors are jumping on the cloud solutions and almost every day new cloud services and providers are seeing the light of day. The velocity of new companies getting created takes memories back to the days before the Internet bubble, maybe we are creating a cloud bubble at the moment.

However, we have seen that from a collapsing bubble also good things can come. When the Internet bubble collapsed the good where separated from the bad, this is something we might also see from a potential collapse of a potential cloud bubble.

Even though the cloud brings a lot of good there are some things to consider when moving to the cloud. I already discussed the legal implications of moving to the cloud in another blogpost. There are however more things to consider. One of the is related to a potential collapse of a cloud bubble. 

One of the things to consider next to the legal impact of cloud computing and the fact that your data might be hosted on a country that has a different legal system then the country where your company is located is the fact of data-silos. The risk of creating a data silo is on the agenda of a lot of the CIO's currently and it is indeed a potential risk of cloud computing.

When we talk about a data silo we talk about an cloud based application where we put in a lot of information however lacks the ability to extract this information. A large number of cloud based companies, and especially SaaS like companies do offer a great way of having your application in the cloud and provide you with options to enter information into this system. However the raw stored data is in most cases not accessible or in a very limited way. This makes that it is almost impossible to leave the cloud vendor and take your data with you to a new vendor. 

In cases where this is possible, based upon a contractual obligation or via build in functionality it often is a hard task to get the data in such a format that it is usable to be migrated to a new system. In cases where you have time to plan for a migration this is often costly however not impossible. In cases where this has to be done suddenly, for example due to a vendor that is bankrupt, you will not have the time to start a project for this. In those cases it might happen that you are unable to extract the data or extract it in a form that it is usable. 

It is good practice to check with your cloud vendor who is offering you a SaaS solution, or even a PaaS or IaaS solution, what your option are to access your data and what services are available to extract this. Next to this it is good practice to check with your vendor on what the rundown policy and exit clauses are in the contract in case the contract is ended and in case of a sudden bankruptcy. 

In an ideal situation your vendor has a escrow like mechanism in place for your data to ensure that your data is still available in case of a sudden bankruptcy. This will prevent you from data loss in such a situation, it will however not prevent you from service loss.

In case you want to extract your data during the contract period or at the end of a contract period it is ideal to have a mechanism in place for data extraction. Such a mechanism can be simple dump of all information in a pre-defined format, a direct coupling to the database holding your data or access to your data via an API.

jDeveloper versions for SOA BPEL extensions

When developing Oracle SOA and BPEL components with Oracle jDeveloper you do want to include the extensions for this  to your jDeveloper environment. When selecting a version of jDeveloper you have to be carefull as not all the versions do support this out of the box. When you are reading the information from the download page of jDeveloper  att he Oracle technology network you will see a warning about this subject at certain 11.1.2.x versions.

Important Note - This version of JDeveloper doesn't include the SOA, WebCenter, and Oracle Cloud development features - to use these components you'll need to download Oracle JDeveloper 11.1.1.6.0.

This indicates you will have to download a 11.1.1 version instead of a 11.1.2 version to be able to run the SOA, WebCenter and Oracle Cloud development features. If you check the release notes of the jDeveloper sherman release (11.1.2) you can find the following statement:

There is no design-time or runtime support for SOA, BPM, and WebCenter components in 11g Release 2.

Oracle has stated that release 2 will be focussing primarily around ADF and where release 1 is focusing around SOA components. This essentially makes it 2 different products both dedicated to there own task. When you like to develop SOA components you will have to select release 1, when developing ADF you have to select release 2. This is if you want to make use of all the best solutions build into jDeveloper.

For people who are developing both ADF and SOA components this is providing a possible issue. According to Oracle this would mean that you will have to install 2 versions of jDeveloper on the same workstation to be able to work with it correctly. Even though this is not a direct issue it is not desirable. According to unconfirmed sources the "issue" should be resolved in the 12C release of Oracle jDeveloper.

There are unofficial ways to get this working within one version however it is not recommended  If you have a release 2 installed on your workstation and are in need to build both ADF and SOA components the best way is to uninstall jDeveloper release 2 and install the latest release 1 which is 11.1.1.6..0

When uninstalling Oracle the best way to remove an already installed version of jDeveloper is by making use of the uninstall utility. A large number of people just delete the installation however jDeveloper comes with a quit OK uninstall option which can be found at $JDEVHOME/utils/uninstall/

The mobile keyboard layout Minuum

As long as there are computers that use a keyboard people have been developing new keyboard layouts. Keyboards differ between countries due to special characters in the language or they differ just because something picked up in a specific country and something else did not. with he growing adoption of mobile phones and tablets at a later stage people have been looking at how to include keyboards on smaller surfaces then your average keyboard. Some inventions have been developed to interact in different ways with the suer. We have seen T9 as a predictive text mechanism, we have also seen swype as a new way of interacting with the keyboard and many others.

Now a new a promising new way of interacting with a keyboard is being developed by Minuum. The Minuum keyboard layout let you type on every surface of any size. Specially the ability to type on a very small surface is important as we see more and more smaller devices come to the market where we do have a need of interacting with.

The below video is showing how the Minuum keyboard is working. Specially the keyboard option attached to the watch is something that is quite nice. If we think about this in combination with a project like Google Glass this can become very interesting in the future.

Face recognition more then only for security

Face recognition is something which triggers immediately recollections to the movie Minority Report for most of us. For some time is has been something from science fiction movies until a couple of years back when we started to find real-world face recognition in our day-to-day life. For people working in highly secured environments they have encountered it when entering restricted areas where a combination of an ID-card and fingerprint and face recognition stated if you could enter a secured area or not.

Today you see more and more face recognition  systems in combination with security. Companies like Artec and Biometry develop products to help companies and governments regulate access to restricted areas.



For the general public face recognition became something in their day-to-day life when social networks started to use face recognition to help you tag friends in pictures when you upload them to a social network. Facebook is one of those social networks that started to use this technology. For implementing it they received a lot of comments from the privacy community and goverment agencies.  On the Facebook blog you can find some more information on the why and how they implemented this option;

Because photos are such an important part of Facebook, we want to be sure you know exactly how tag suggestions work: When you or a friend upload new photos, we use face recognition software—similar to that found in many photo editing tools—to match your new photos to other photos you're tagged in. We group similar photos together and, whenever possible, suggest the name of the friend in the photos.

Face recognition is a controversial field and facebook has received a lot comments on using this feature and governments and privacy organizations have been pushing to remove this option. In some cases they have succeeded and Facebook has been forced to disable the feature and remove the data for face recognition as can be found in this article of the Daily Mail.

In general face recognition makes use of principal component analysis to analyze a face and match this to a set of data stored in a database. Principal component analysis (PCA) is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. The number of principal components is less than or equal to the number of original variables. This transformation is defined in such a way that the first principal component has the largest possible variance (that is, accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest variance possible under the constraint that it be orthogonal to (i.e., uncorrelated with) the preceding components. Principal components are guaranteed to be independent only if the data set is jointly normally distributed. PCA is sensitive to the relative scaling of the original variables. Depending on the field of application, it is also named the discrete Karhunen–Loève transform (KLT), the Hotelling transform or proper orthogonal decomposition (POD).

However, next to the fields of security and a help in tagging friends in pictures there are some other fields in which this is used. For example security forces use it to find and track people in public places, casino's as well use it to find people who cheat and pick them out of the visitors when they visit a casino again. Software vendors are using it to replace the standard username and password authentication for operating systems and phone's. We are most likely to much more coming our way as developers do have now access to opensource implementations of face recognition software via the OpenBR project. 

A new way of using the algorithms to recognize a face is an implementation done by the people from Ditto Technologies Inc. Ditto has developed a system that scans your face and find the perfect matching glasses for you based upon the biometric information received from a scan of your face.

As we can see on the Ditto site, face recognition and the associated algorithms might be able to help us in other fields we did not think of yet at this moment. Picking glasses is just one of them. Looking at current tech startup companies we can see quite a lot of companies who are doing something with face recognition and this field will grow in the future even more.

big data visualisation

A large part of the effort in developing solutions around bigger data are concentrating around retrieving data, storing data and getting meaning out of this data. For a lot of engineers and developers getting meaning out of the vast amount of data is the end of the task. The system, for example a Hadoop implementation, has crunched vast amounts of data and is providing the answer in the form of a array of data or a file of consolidated answers. Coming from a large pile of structured and unstructured data this is a huge accomplishment however it is not the end of things.

The next step and in a large number of cases is getting this set of data, which is the result of your big data analysis, to the users. To be more precise, how do we get it there in a usable form. Receiving a flat text file is not always the best way of delivering your results to a user or customer. This might be the correct way to provide it to an analyst who will do more analysis on it however for a user or customer who quickly wants to see the information this is not the correct way most likely. 

One of the fields of big data and the way we work with computers and computer systems in relation to data will be big data visualisation. Big data visualisation will be working focusing around how do we handle massive amounts of data and how do we make sure this is represented in such a way that it is understandable by a human. Big data and the vast amount of data that is generally associated with big data solutions is no longer interpretable by a human, that is why we have machine who do this for us. However now we have to focus on how to ensure that the outcome of the big data analysis is visualize in a human interpretable format.

Below a high-level diagram is showing a setup for a distributed sensor network. Showcasing in this diagram are the sensors picking up signals and transmitting them to the sensor server in a M2M (machine to machine) fashion. This completely is part of the "sensor Domain" in the diagram. The sensor server is placing the retrieved data in to the Hadoop cluster by placing it on the HDFS (Hadoop Distributed File System) within the Hadoop Domain. 

Data within the Hadoop domain is often referred to as the data lake. Big data stored in HDFS, or as stated the data lake, is at this moment unstructured data. By using MapReduce on the data placed by the sensor server within HDFS we can make more meaning of this unstructured data. This is commonly where the point where most developers on Hadoop and MapReduce will stop. Providing the consolidated and structured data and answers via MapReduce is the endpoint. 

The next step however is presenting this data to the users and customer or to a analyst team who will do  more analysis on this data. For this it is needed to load the results of your MapReduce job into a database. In our case as shown above this is a database within the BI Domain.  The structured data resulting out of the MapReduce job can now be used within other applications or BI tools to be presented in a human readable way. 

When you have a dominant Oracle landscape it can make sense to deploy Oracle BI as a tool to connect to the database holding the MapReduce results. One of the benefits of this is that OBIEE will also allow you to use multiple datasources. Meaning that you can use the MapReduce results coming from your sensor network in combination with for example you sales data in the Oracle eBS database. OBIEE is providing some great ways of visualising data, it gives your users and customers a view of the results in multiple ways which are very human readable. 

However, when using Oracle BI tools you are limited to the options provided by the tool, this is the same for most of the tools coming from other Vendors. What I expect we will see in the upcoming future is the rise of new companies who specialise in data visualisation. One of the example of such a company is Periscopic who specialise in this field. Also we do see a lot of online companies and services coming to live providing support in visualising data. 

Also a lot of good opensource libraries are coming to life where you can download (and contribute) to open ways of visualising sets of data. A very cool implementation of opensource data visualisation if D3JS. Some examples of data visualisation done by making use of D3JS are shown below.

Oracle enterprise reference architecture

Everyone who has been watching Oracle and the strategy from Oracle closely will have noticed that they have been growing by acquiring new products and companies and by developing new solutions. When you look at the current portfolio of Oracle you will notice that they can provide solutions in a wide range of fields. One of the daunting tasks for Oracle is providing an overall strategy and architecture to combine all those products and product lines into a reference architecture which can be used and deployed by architects at there customers and at system integrators. To help architects with this Oracle has developed the "IT Strategies from Oracle" or in short ITSO. ITSO helps customers to find there way in the wide range of products and solutions and build IT strategies and architectures based upon the Oracle best practices and blueprints.

The general starting point for ITSO is the overview image of the IT Strategies from Oracle from which you can drill into details. 

As you can notice in the above overview image the IT strategies from Oracle do consist out of 3 main components. These are: "Enterprise Technology Strategies", "Oracle Reference Architecture" and "Enterprise Solution Designs". To fully understand the the IT strategies from Oracle it is of vital importance to understand the meaning of the three main components and there role within the overal strategie from Oracle. 

Enterprise Technology Strategy

The Enterprise Technology Strategies or in short ETS is the integration of the vertical positioned technologies into the core part of the ITSA, the core part is the ORA within ITSA. Within the technologies mentioned in ETS you can find technology paradigms like SOA, BPM, EDA, etc. From within the ORA there is a connection made to the verticals within the Enterprise Technology strategy. 

Oracle Reference Architecture

The Oracle Reference Architecture or in short ORA. ORA is a logical architecture reference showcasing how to build and deploy enterprise class solutions. The purpose of ORA is to provide a reference architecture for designing, building, and integrating solutions based on modern technology from Oracle and other vendors. The reference architecture offers architecture principles and guidance based on recommendations from Oracle product development architects and field experts. Information provided by ORA gives architects an understanding of how to design solutions for the Oracle environment and best leverage its capabilities.

It is important to note that One of the key concepts of ORA is to be vendor neutral  and product agnostic. This means that within theOracle Reference Architecture core document no Oracle products are specifically promoted. Having stated this, there is an exception made by Oracle that they will name specific vendors and products in case that this is needed to clarify certain principals or to be used as an example.

Enterprise Solution Designs

The Enterprise Solution Designs or in short ESD is used to provide insight in specific industrie  perspective. It combines specific industrie requirements into solution designs  which leverage the information in the Oracle Reference Architecture and in the Enterprise Technology Strategy documents.

The entire stack of information provides architects working with Oracle products and/or developing architectures for enterprise grade solutions. This information can be valuable to craft a overall architect strategy which you can follow and use as guidance in creating a strategic roadmap for the future and set design principles for a team of architects. 

Social and location based marketing

The combination of social data and mobile location data can be a very big game changers for advertising. Combining the insight of what a person his interestes are and the location can help to promote new products and services in a very targeted fashion. To be able to retrieve the amounts of data needed and to process this data you will have to look at big-data solutions. In the below video IBM is showcasing the promotion of a new coffee place next to a theater. You can see how a specific group of people is selected based upon there location and based upon the interests that they express via social media.



Big data is creating opportunities for Communications Service Providers (CSPs) to establish new revenue streams. With big data technology, CSPs can analyze the location data generated by millions of mobile devices and use the resulting insights, along with offerings from business partners, to deliver customized services and offers that consumers want.

IBM is getting currently more and more attention on there big-data solutions and thinking. Specially for big-data IBM has created a specific big data hub site which can be interesting for everyone interested in big-data solutions and how to use them in a enterprise.

Oracle SQL in Eclipse

The Eclipse project started as a project for developing a development platform for Java code. Even though it is still focused arround Java a lot of additions have been build around Eclipse. Eclipse is as the project like to explain; "a community for individuals and organizations who wish to collaborate on commercially-friendly open source software. Its projects are focused on building an open development platform comprised of extensible frameworks, tools and runtimes for building, deploying and managing software across the lifecycle. "

You can find plugins for large number of languages and for a large number of development frameworks that can be enabled within Eclipse. One of the companies investing in the development of Eclipse plugins is Oracle and as Oracle is a database company they also made sure some code was donated to make Eclipse a SQL development environment. This however is somewhat unknown as Oracle mostly promotes the Oracle SQL Developer solution. However, when you only need to work on SQL code occasionally and do most of your development work in Eclipse this can be very handy. Or,... if you just like Eclipse more then you like Oracle SQL Developer.

To start using the Oracle database plugin for Eclipse you will to undertake the following steps.

Step 1:
Start Eclipse and go to "Windows" - "Open Perspective" - "Other"

Step 2:
Selecting the option from step 1 will provide you with the screen as show below. Here you can select the perspective for database development.

Step 3:
The action performed in step 2 will open a new side menu on the left side of Eclipse and will show as the "Data Source Explorer".  At the Database connections menu right-click and select New.

Step 4:
Eclipse is providing a lot of database connection types. If you have installed the Eclipse Oracle additions you will also have the option for an Oracle database. Out of the box a lot of other database connection types are also provide as you can see in the below screenshot.

Step 5:
When you have selected the Oracle database option you will be provided with the below menu where you will have to enter de connection details for this specific database connection.

When you have successfully completed the actions in step 5 and have clicked finish you should now have a working connection and the options to start developing PL/SQL code and start executing commands against your Oracle database. Below is a screenshot showing your Eclipse Oracle SQL worksheet.

Prepare Oracle Linux for EM12C agent deployment

When you are deploying new Oracle Enterprise Manager agents on Linux hosts to monitor your servers it is good to know that there are some security related things that need to be changed to be able to install the agent correctly. During the installation it will be needed that the account on the target server is able to do a sudo.

To enable this you will need to add the account you will be using to install the agent to the sudo users. In our case we are going to add the Oracle Enterprise Manager agent to a hadoop node which has a user named hadoopnode0 under which the agent will be deployed. To be able to install the agent with this user we define a named credential in Oracle Enterprise Manager and state that it needs to execute its commands as the user root.

Before we continu we have to be sure that the user hadoopnode0 is added to the sudoers. For this you will have to edit the file /etc/sudoers and make some changes. Remember that this example is quite a quick and dirty example to get the agent on the host, in a large scale deployment you might want to consider an other strategie with specific usernames and specific sudo rights to it.

First change the following line so it is commented out:

Defaults    requiretty

into this:

#Defaults    requiretty

Next tho this we have to enable the option for visiblepw. This is by default disabled so we need to make sure it is enabled. Considering you have a fresh install change the following line:

Defaults   !visiblepw

Into this:

Defaults   visiblepw

Now add the user that you will be using to the file. In our case this is hadoopnode0 and we add a line for this user directly under the line for the root account.

root    ALL=(ALL)       ALL
hadoopnode0     ALL=(ALL)       ALL

After changing those settings you should be able to use the agent deployment options on the host in Oracle Enterprise manager. More details on how to deploy an agent can be found in the Oracle Enterprise Manager add discovered host blogpost

Big-data for the internet of things

If you look at the future outlook of the technology industry you will notice that more and more analyst firms are talking about "the Internet of things". The Internet of things is the conceptual believe that in the future more and more devices will be connected to the Internet. This will include not only the devices that we currently connect to the Internet like workstations, smartphones, tablets and even audio equipment and television sets. In the conceptual idea of the "Internet of things" the idea is that we connect all electronic devices to the Internet. This will include your fridge, your washing machine, your lawn mower and even your light bulbs in your house.

The idea is that all devices should be able to communicate back in a calling-home principle and that you interact with them remotely. This is providing some great new opportunities for both the consumer as well as the producer of the equipment and others. Consumers will be able to interact differently with there equipment and will be more informed about things without the need to actively gather the information.  Producers will be receiving feeds of information about the use, mis-use and malfunctioning of the equipment they have sold and can use this to tweak new products more on real usage.

Some people will be in favor of such a "Internet if things" some will be against it. We will see legal battles on who is the owner of the data you send out via your device, questions around privacy and questions around liability. Whatever the outcome of those legal discussions we will see the "Internet of things" coming to out houses and we will start using it at some point.

The interesting part in the concept of the Internet of things is not purely the new products that will be developed and the new business opportunities that will arise from it. It will be for a large part about the new technologies that need to be developed to make the Internet of things work. If you keep a close watch on what is happening within the technology field you already see some things happening. Some of them are prerequisites for the Internet of things some are enablers and some are direct part of the concept.

One of the prerequisites for example is IPv6 a new version of the IP protocol. This is a new version of the protocol which allows more addresses to be assigned. For many Internet users this will be something they are not aware of however it is one of the prerequisites for the Internet of things.

A second movement in the technology industry is the rise of big-data. Not so much the rise of data itself however more the new way of how we handle the massive amounts of data. As we will be adding more and more devices to the Internet and they will be sending out data to consumers and to producers there will be the need of handling this vast amount of data in way that we can make sense of it. Having the option to handle vast amounts of data will be a key factor to success for businesses.

In the "past" big-data was the field of technology driven companies, companies like google, yahoo, Facebook and twitter where the early adopters and inventors of how to handle massive amounts of data. The new players in this field who need to beta grip on how the need to handle vast amounts of data will be companies who are not considered high-tech companies in the IT field. This will be for example companies who produce washing machines. Where in the past they needed to be experts in mostly mechanical and electric engineering ow they need to play there part in data handling.

If for example your washing machine starts calling back home to the producing company about all the sensor data it can produce this will be enormous. Think about sensor data of temperature, rounds per minute, water used, amount of clothing, menu selection by the user. All this data is of great importance to the producing company as they can turn the real world into a testing lab.

This provides a couple of challenges for such a company. How are they going to receive this information which is coming to them in a high velocity, how to store it, how to do analysis on the stored data and how are they going to get value out of it for both the company and the consumer.

Understanding how to handle big data coming for sensors and devices developed companies who normally are not into high-tech will be a key to success for those companies. The rise of the "Internet of things" will bring a rise in new technologies around data and within the need for experts in this field.

Understanding Gartner research methodologies

Gartner is one of the world's leading information technology research and advisory company. Commonly Gartner reports are read by CxO level executives and decision makers and they provide a good view of what is happening in the world of technology at this moment and what the upcoming trends are. The current Gartner market research is focussed around a number of methodologies. Understanding the methodologies can be important. It can be important if you are a technology vendor and Gartner is conducting research in your field. Due to the common audience of Gartner reports it can be a game changer if Gartner positions you favourable.

Next to this understanding the Gartner methodologies can be important if you are a CxO executive and want to make a decision and would like to make use of the reports from Gartner. Understanding the reports correctly and interpreting them correctly is in this case of vital importance.

A third group of people reading the Gartner reports are market analysts, technology analysts and entrepreneurs. Gartner reports are helping you to understand the current technology market and the way emerging technologies are heading.

Gartner describes it's methodologies as follows;
Our proprietary research processes allow us to see IT as it pertains to the evolving business landscape. Our research methodologies are based on our years of experience observing trends and scientifically mapping technology's progress against true delivery. 

 The success of our proprietary methodologies lies in distilling large volumes of data into clear, precise, actionable insight and advice so our clients can formulate plans or make difficult business decisions. 
The observations and recommendations delivered through our proven methodologies ensure that you make decisions about the business applications of IT with higher levels of confidence. 

Our clients make important multi-year technology investments in the face of uncertainty and risk. Our methodologies help our clients reduce and manage that risk, and enable them to succeed in their roles as they mobilize IT to contribute to their organization's business objectives.

The main research methodologies used by Gartner are; Hype Cycles, Magic Quadrant, MarketScopes, ITScores, Vendor Ratings, Market Forecasts, Market Share Analysis and IT Market Clocks. All are proprieatary to Gartner.

Hype cycles:
When new technologies make bold promises, how do you discern the hype from what’s commercially viable? And when will such claims pay off, if at all? Gartner Hype Cycles provide a graphic representation of the maturity and adoption of technologies and applications, and how they are potentially relevant to solving real business problems and exploiting new opportunities. Gartner Hype Cycle methodology gives you a view of how a technology or application will evolve over time, providing a sound source of insight to manage its deployment within the context of your specific business goals.

Magic Quadrant:
Who are the competing players in the major technology markets? How are they positioned to help you over the long haul? Gartner Magic Quadrants are a culmination of research in a specific market, giving you a wide-angle view of the relative positions of the market's competitors. By applying a graphical treat­ment and a uniform set of evaluation criteria, a Gartner Magic Quadrant quickly helps you digest how well technology providers are executing against their stated vision.

MarketScopes:
When markets are growing and IT solutions are stable, Magic Quadrants provide the best tool for understanding how the players are competitively positioned. But when new markets emerge and user requirements are in flux, solutions are often approached in wildly different ways, making a competitive positioning less useful. Mature markets present a similar challenge, as the differentiators among consolidating technology providers and solutions grow more difficult to discern.

ITScores:
Gartner ITScores are holistic sets of interactive maturity assessments designed to help CIOs and IT leaders evaluate the maturity of both the IT organization as a provider of IT services, and the enterprise as a consumer of information technology. Unlike other IT maturity assessments, a Gartner IT Score measures your organization's capabilities within the context of an enterprise culture, behaviors and capacity for leadership - factors that dramatically impact IT's effectiveness and it's ability to contribute real business value.

Vendor Ratings:
Clients use our well-defined methodology to rate IT technology providers—large, small, public or private. Gartner Vendor Ratings assess all the different aspects of a technology provider, such as its strategy, organization, products, technology, marketing, financials and support. These ratings are periodically revised to reflect changes in assessment when a significant internal or external event directly affects the provider.

Market Forecast:
How do you quantify the impact of a market’s business drivers? Is a market growing, retreating or flat? The best answers come from an analysis of both supply and demand, comparing technology investment trends in the provider community with the spending trends of end users. It’s a unique picture you get from Gartner, based on our relationships with thousands of end users and providers. And it’s a view you can’t get anywhere else.

Market Share Analysis:
Understanding market share is one of the most important metrics used by executives in any business. Through our Market Share Analysis methodology, clients see how share is allocated among 400 technology providers in 37 key markets. Our detailed analysis of how provider revenue is allocated reveals what types of solutions are succeeding, which are trailing and where opportunities exist for providers to take additional share.

Market Clocks:
The useful life of every technology product or service has an end, beyond which it will be more cost-effective to retire and replace the asset than to continue maintaining it. Gartner IT Market Clocks are decision frameworks that provide a full life cycle view of technology assets - whether capabilities, products or services. They help you better evaluate the technology assets you are responsible for, so you can prioritize IT investments and build technology road maps that support business plans.

How Linux is built

Linux is already for years a big player in the server market. A large number of servers running on Linux help millions and millions of people in there daily lives without even knowing that they interact with Linux. More recently Linux is coming in phones and in devices. More and more workstations are becoming Linux or Linux based. It is already for years no longer the exclusive domain of developers and coders. A good moment to look back at how the Linux kernel and your linux solutions are build and where they come from. In the below video you will get a quick introduction, the video is made by TheLinuxFoundation.