Thursday, May 19, 2016

Oracle Linux - Clone file permissions with chmod

Every now and then file permissions under Linux can be tricky. in some cases a wrong file permission can make it happen that things do not work they way you would expect them. Also I found that a lot of people find it challenging to set the correct file permissions using the command line under Linux. A way to make life more easy in some cases is to use the option to "clone" file permissions with a single command.

For example, if you have created some addons to a tool running on an Oracle Linux system and you want to the addon file to have the same permissions as another file you can use the --reference option from the chmod command.

As an example we have two .jar files:

[root@localhost ~]# ls -l *.jar
-rwxr-xr-x. 1 root root 88 May 19 10:48 addonExecution.jar
-rw-r--r--. 1 root root 11 May 19 10:48 executionLib.jar

We want to make sure that the addonExecution.jar has exactly the same permissions as the executionLib.jar file. We can do this by specifying the desired stated in a chmod command, we can also use the --reference option as shown below:

[root@localhost ~]# chmod --reference=executionLib.jar addonExecution.jar

This will make sure that the addonExecution.jar file now has exactly the same permissions as the file used as a reference.

[root@localhost ~]# ls -l *.jar
-rw-r--r--. 1 root root 88 May 19 10:48 addonExecution.jar
-rw-r--r--. 1 root root 11 May 19 10:48 executionLib.jar

Another use case example of this is that you can use it in a bash script where you might not be sure what the permissions should be for a certain file and only know that they always need to be the same as a specified other file. By using the --reference option you do not explicitly need to know the permissions during the creation of the bash script, you only need to know which file can be used as a reference. 

Oracle Linux - remove duplicate lines with awk

Sometimes you want to clean data quickly and remove all duplicate lines that are present in the file. For example a raw output from a system that is "dumped" on your Linux file system needs to be cleaned before you use it as input into another system. You can write some fancy code to do so, you can also use a very simple and straight forward solution by using awk on your Oracle Linux bash shell.

In the below example we have a file (the data with the duplicate lines) called rawdata.txt and we want to make a clean file called cleandata.txt. The example awk command can be used to read rawdata.txt and write the clean data to the file cleandata.txt

awk '!seen[$0]++' rawdata.txt >> cleandata.txt

The command itself is a very quick and dirty solution, most likely you want to use this in a wider script that is cleaning your data in a more sophisticated manner. 

Sunday, May 15, 2016

Oracle Linux Name Service Switch libraries

When scripting a bash solution which needs to check if a user is existing on your Oracle Linux instance you have a couple of options. The most known solution is to check if the username is present in the /etc/passwd file. You can simply do a cat of this file and using grep and wc command to make it more usable in your script. An example of this could be for example the command below which will give you the number of times that “apache” is mentioned in the file. Do remember, we assume this is the user apache and this is not very reliable in reality.

cat /etc/passwd | grep apache | wc –l

Another solution is making use of getent which is not that well known as the above example. The getent command displays entries from databases supported by the Name Service Switch libraries. An example of this is shown below:

[root@dev1 ~]# getent passwd apache
apache:x:48:48:Apache:/var/www:/sbin/nologin
[root@dev1 ~]#

Where in case the user is not existing the command will provide no output:

[root@dev1 ~]# getent passwd apache222
[root@dev1 ~]#

Using a wc –l on getent will provide you a more pure answer opposed to a wc –l on a cat from the passwd file. As stated; The getent command displays entries from databases supported by the Name Service Switch libraries. To understand this in a bit more detail and understand what databases are that are supported by the Name Service Switch libraries you can check the configuration file. Under Oracle Linux (and most other Linux distributions) this can be found at /etc/nsswitch.conf .  An example of a standard nsswitch.conf file is shown below. As you can see a lot more is supported by the Name Service Switch libraries and not only passwd.

#
# /etc/nsswitch.conf
#
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
#
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
#
# Valid entries include:
#
#       nisplus                 Use NIS+ (NIS version 3)
#       nis                     Use NIS (NIS version 2), also called YP
#       dns                     Use DNS (Domain Name Service)
#       files                   Use the local files
#       db                      Use the local database (.db) files
#       compat                  Use NIS on compat mode
#       hesiod                  Use Hesiod for user lookups
#       [NOTFOUND=return]       Stop searching if not found so far
#

# To use db, put the "db" in front of "files" for entries you want to be
# looked up first in the databases
#
# Example:
#passwd:    db files nisplus nis
#shadow:    db files nisplus nis
#group:     db files nisplus nis

passwd:     files
shadow:     files
group:      files

#hosts:     db files nisplus nis dns
hosts:      files dns

# Example - obey only what nisplus tells us...
#services:   nisplus [NOTFOUND=return] files
#networks:   nisplus [NOTFOUND=return] files
#protocols:  nisplus [NOTFOUND=return] files
#rpc:        nisplus [NOTFOUND=return] files
#ethers:     nisplus [NOTFOUND=return] files
#netmasks:   nisplus [NOTFOUND=return] files

bootparams: nisplus [NOTFOUND=return] files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

netgroup:   nisplus

publickey:  nisplus

automount:  files nisplus
aliases:    files nisplus

Wednesday, May 11, 2016

Oracle Linux addons channel

When working with Oracle Linux and using the Oracle Linux YUM repository you might be missing out. It is worth while checking which channels are being activated for YUM as not all channels are active by default after an installation or might have been disabled by a template installation.

Something I do see commonly is that administrators who create Oracle Linux templates do disable a lot of the YUM channels. If you receive a template based installation in a private cloud setup it is good practice to first check which channels are activated and which have been disabled. If some of the channels, for example the addons channel, are disabled you might be missing out on a lot of things and find yourself in a “dependency hell”

Rather than trying to resolve this manually it is good to check the channel settings in /etc/yum/repos.d/* normally this will be /etc/yum.repos.d/public-yum-ol6.repo for OL6 machines.

A default installation will state enabled=0 for the public_ol6_adddons channel. To enable this you will have to state enabled=1

Even though it might sounds like a minor change, it is commonly overlooked in first instance. Additional channels are available for Oracle Linux from the public Oracle YUM server which might be of interest to administrators to enable.

Tuesday, May 10, 2016

Oracle Linux pre-install RPM for EBS

When deploying Oracle software on a Oracle Linux (or any other Linux distribution) you will have to set a number of pre-requisites. In some cases you need to set certain kernel parameters and ensure you have specific packages installed on your machine.

For the Oracle database there is already for a long time a way of doing this by installing a specific RPM. As this is providing a great way of preparing your environment for the software installation Oracle is also providing a similar mechanism for preparing your environment for the installation of Oracle e-Business suite.

You can install a pre-install RPM for Oracle e-Business Suite 12.1 and 12.2 which you can fetch from the addons channel at ULN. A best practice for creating a new Oracle Linux environment to run Oracle e-Business Suite is to do a minimal installation of Oracle Linux and install the pre-install RPM on top of this. Reason for this is that you will not have unneeded packages and functionality which might cause hindrance at a later stage.

  • In essence the pre-install RPM will undertake the following tasks:
  • Downloading and installing all software package versions and dependencies required for installing E-Business Suite R12 (12.1, 12.2)
  • Creating the users oracle and applmgr for use as owners of the database and application tiers respectively, while setting hard and soft shell resource limits
  • Updating kernel parameters in /etc/sysctl.conf to recommended values
  • Sets DNS resolver parameters in /etc/resolv.conf to minimum recommended values
  • Sets 'numa=off' in the kernel in the kernel commandline
  • Disables 'Transparent Huge Pages (THP)' if enabled 



For more information on the pre-install RPM’s please refer to the following notes on My Oracle Support: 761566.1 & 1330701.1

Tuesday, April 26, 2016

Oracle Hybrid Cloud

Recently I presented together with Marcel Giacomini from Oracle on Oracle public, private and hybrid cloud. The hybrid cloud is a direction I personally feel the market will move towards very quickly. Even though cloud companies would like to see enterprises adopting a full cloud model I think a majority of the large enterprises and companies will take the route of hybrid cloud first.

To see more on the capabilities around hybrid cloud from Oracle have a look at the deck we presented during Advantage You.


Wednesday, April 20, 2016

Oracle Linux Unsupported Packages

When running Oracle Linux you do not have to purchase a support contract from Oracle. You are perfectly fine running Oracle Linux without purchasing the support. However, in general, when running Oracle Linux in a business environment you would like to have the option to call in support when needed. This means that most companies do purchase the support and use it whenever needed. A general misunderstanding is that everything shipped by Oracle is also supported by Oracle.

In fact some (a limited) parts are not supported by Oracle while at the same time you will be able to find them in the Oracle Linux distribution and you have the option to install them and use them. The general misconception comes from the fact that most people understand that when you download and install additional software that is not provided by Oracle you will not get support. At the same time they expect everything shipped by Oracle to be under the support contract.

In case you are in doubt if a specific part is under support you might want to check the “Unsupported Packages from ISO” at the Oracle Linux website. This list (current date – Do check the latest version) has  the following packages:

  1.  ccs
  2.  cluster-cim
  3.  cluster-glue-libs-devel
  4.  clusterlib-devel
  5.  cluster-snmp
  6.  cman
  7.  cmirror
  8.  cmirror-standalone
  9.  corosynclib-devel
  10.  ctdb
  11.  ctdb-devel
  12.  dlm
  13.  fence-agents-all
  14.  fence-virtd-checkpoint
  15.  foghorn
  16.  gfs2-utils
  17.  haproxy
  18.  ipvsadm
  19.  keepalived
  20.  libesmtp-devel
  21.  luci
  22.  lvm2-cluster
  23.  lvm2-cluster-standalone
  24.  N/A
  25.  omping
  26.  openaislib-devel
  27.  pacemaker
  28.  pacemaker-doc
  29.  pacemaker-libs-devel
  30.  pcs
  31.  piranha
  32.  python-repoze-what-quickstart
  33.  resource-agents
  34.  rgmanager
  35.  ricci
  36.  xfsdump
  37.  xfsprogs
  38.  xfsprogs-devel


Monday, March 28, 2016

virtualbox only showing 32 bits options

I received  my new laptop from my work this week. In general I tend to be not that happy with receive a new laptop from work because it takes some time from your day to get everything back working again. However, this time I was more disappointed than normal as it turned out that I was not able to run 64 bit guests on my laptop and only 32 bits options where available.

After some checking I found out that my OS was a 64 bit OS and everything should work as far as I could see. However, only 32 bits options where available. As it turns out Windows 7 in combination with virtualbox is not allowing you to run 64 bit guests when certain virtualization is not enabled in the bios of your machine.

After turning on "Intel Virtualization Technology" and "Intel VT-d feature" virtualbox again allowed for running 64 bit guests.


Just a small reminder for everyone who runs into this issue. In case you have this enabled and it is not working, make sure that you disable Hyper-V on windows.

For more background information on this you can refer to ticket 12350 on the virtualbox website. 

Sunday, March 06, 2016

Oracle Linux - Build a Private YUM Server

Most people working with Oracle Linux will have a way of updating systems by using YUM. When using Oracle Linux as a personal workstation or just a private test server it is perfectly good to use the public YUM server from Oracle.

In cases where you have a secured environment running production servers you most likely do not want to use a connection to a external system to get your updates. You would like to have a local copy of the YUM server. In those cases you can create a satellite YUM server. This will make sure that you always have an updated repository locally which is in sync with the public yum server from Oracle.

However, a second reason can exists to have a local YUM server. In cases where you develop your own RPM's and want to make them available to your Oracle Linux machines it is good practice to run your own local YUM server.

Building a local YUM server is quite easy and not a hard task, making the short investment to ensure you have a your own local YUM server for your self developed RPM's or for packages from other vendors needed in your landscape makes much sense.

The below diagram shows on a high level on how your deployment can look like, not including all firewall components and network components you should use to ensure a correct network zone model for security.

Create YUM Repository
Creating a YUM repository takes a number of steps. First of all you need to have a location where you will store the RPM's and the associated repository meta-data. In our case we know we want to make the YUM repository available via HTTP (using NGINX) at a later moment so we create a location like shown below for our company repository.

  mkdir /var/www/repos/companyrepo

The next step will be to ensure that you copy all the RPM's you want to include in your company repository to this location. As soon as they are moved to this location you will have to make this into a real repository. For this you will need some tooling installed on your server. You can do so by installing the below RPM's

  rpm -ivh deltarpm-3.6-3.el7.x86_64.rpm
  rpm -ivh python-deltarpm-3.6-3.el7.x86_64.rpm
  rpm -ivh createrepo-0.9.9-25.el7_2.noarch.rpm

As soon as you have the required tooling installed you can create a repository by executing the following commands:

  createrepo

That's it? yes, that is it. Nothing more to do. It will go through your RPM's stored and will create the needed information. You will find a new director named repodata which contains all the data needed for remote YUM clients to connects

Configure NGINX
Having your RPM's and the YUM metadata in the newly created repodata is great. However, what you want is to ensure your clients (servers) can connect to the YUM repository (in the diagram located on yum.company.com). To enable clienst to connect you will have to ensure that the location where you store this (in this example /var/www/repos/companyrepo) is available via HTTP.

As we do not need a heavy weight HTTP server we can use for example NGINX. To install NGINX on Oracle Linux you can refer to this blogpost which provide the needed guidance.

As soon as you have NGINX running you need to ensure that it will point to /var/www/repos/companyrepo . As an example you will need to undertake the following steps to complete the configuration of NGINX.

1) Create a file named companyrepo.conf in /etc/nginx/conf.d
  touch /etc/nginx/conf.d/companyrepo.conf

2) Edit the file with VI and ensure you have the below information in the file:
  server {
      listen       80;
      server_name  localhost;

      location / {
          root   /var/www/repos/companyrepo;
          index  index.html index.htm;
      }

      error_page   500 502 503 504  /50x.html;
      location = /50x.html {
          root   /usr/share/nginx/html;
      }
  }

3) Test if you can access this location on yum.company.com with a standard browser.

Configure Local Servers
As we now have a local YUM server running on yum.company.com and made it available via HTTP we can now configure the clients (servers) to connect to the local YUM repository and make use of it.

Make sure you have the below configuration in a .repo file in /etc/yum.repos.d . You can have other files in this location as well if required however the below content in a .repo file (for example companyyum.repo) will ensure you can connect to the local company YUM repository and use YUM for installing your own RPM's or RPM's from other vendors.

[COMPANYYUM]
   name=Local company YUM repository
   baseurl=yum.company.com/
   gpgcheck=0
   enabled=1


Update YUM Repository
In case you need to update your local yum server and want to add RPM's to it you do not have to go through the enture createrepo procedure. This can, in case you have a very large set of RPM's, take quite some time. In case you only want to add the new files to the YUM meta-data and make them available to users you can issue the below command instead.

  createrepo --update .

Now your server has the new RPM's availabel for YUM users to connect. In case you directly issue a yum command on one of your clients and it is unable to find the new package it might very well be that your client is having a cache of the repository data locally. This will not contain the new package. To make sure your client is doing a fresh grab of the repository data you will have to enforce the expiration of the local cache by issuing the below command on the client:

  yum clean expire-cache

The time the cache is kept is configured in /etc/yum.conf and by default it reads keepcache = 1 . This means that you might not run into this issue on every client. However, if your client already did an interaction with the YUM server that day it might have a cache that is not expired yet and might not find the package due to that reason. 

Wednesday, February 24, 2016

Oracle managed file transfer

Whenever someone asks a Linux administrator that a file will be generated on a regular basis and this needs to be transferred to another location, another server or even to another company the solution is commonly that a small bash script will be created. Over time one small bash script will become two scripts, will become a ten scripts and then become an unknown and undocumented number of scripts. Created one by one in an organic growth model. In essence we have to realize that even though this is a quick and dirty solution which is often being practiced this is not the correct solution.

When you are seriously looking for a managed way of securely transferring files you cannot rely on a number of bash scripts being started by cron. You will need to have a more solid solution. Oracle provides a solution from this in the form of MFT or Oracle Managed File Transfer. I recently wrote a paper on this which can be found on this site. Also a short slidedeck can be found on my slideshare page.

And, if you want a quick a dirty intro with a video, you can find one below.

And, even though I love to create a bash script under Linux and use this for my own systems at home, I have to admit that using a solution like that is not something you really want in a production environment. In case you really need to securely move files in an enterprise environment the MFT solution is something you should look at. 

ELBA-2016-0177 Oracle Linux 7 coreutils bug fix update

Oracle Linux Bug Fix Advisory ELBA-2016-0177


The following updated rpms for Oracle Linux 7 have been uploaded to the Unbreakable Linux Network:




Description of changes:

  • [8.22-15.0.1.1]- clean up empty file if cp is failed [Orabug 15973168]
  • [8.22-15.1] - cp: prevent potential sparse file corruption (#1285365)


ELBA-2016-0198 Oracle Linux 7golang-github-cpuguy83-go-md2man

Oracle Linux Bug Fix Advisory ELBA-2016-0198

The following updated rpms for Oracle Linux 7 have been uploaded to the Unbreakable Linux Network:




Description of changes:

  • [1.0.4-2]- Build it for z-stream  related: #1300321
  • [1.0.4-1]- Rebase to 1.0.4   Deps import separatelly, not in one tarball resolves: #1300321
  • [1-5]- Update the spec file for RHEL, Remove devel subpackage, Bundle github.com/russross/blackfriday and github.com/shurcooL/sanitized_anchor_name into tarball, Use bundled dependencies to build md2man  resolves: #1211312
  • [1-4]-  Bump to upstream 2831f11f66ff4008f10e2cd7ed9a85e3d3fc2bed related: #1156492
  • [1-3]- Add commit and shortcommit global variable related: #1156492
  • [1-2]- Resolves: rhbz#1156492 - initial fedora upload, - quiet setup, - no test files, disable check
  • [1-1]- Initial package

ELBA-2016-0220 Oracle Linux 7 libvirt bug fix update

Oracle Linux Bug Fix Advisory ELBA-2016-0220

The following updated rpms for Oracle Linux 7 have been uploaded to the Unbreakable Linux Network:


  • x86_64: libvirt-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-client-1.2.17-13.0.1.el7_2.3.i686.rpm
  • x86_64: libvirt-client-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-config-network-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-config-nwfilter-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-interface-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-lxc-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-network-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-nodedev-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-nwfilter-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-qemu-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-secret-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-driver-storage-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-kvm-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-daemon-lxc-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-devel-1.2.17-13.0.1.el7_2.3.i686.rpm
  • x86_64: libvirt-devel-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-docs-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-lock-sanlock-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • x86_64: libvirt-login-shell-1.2.17-13.0.1.el7_2.3.x86_64.rpm
  • SRPMS: http://oss.oracle.com/ol7/SRPMS-updates/libvirt-1.2.17-13.0.1.el7_2.3.src.rpm


Description of changes:

  • [1.2.17-13.0.1.el7_2.3] - Oracle files:docs/et.png Replace docs/et.png in tarball with blank image
  • [1.2.17-13.el7_2.3]- vmx: Adapt to emptyBackingString for cdrom-image (rhbz#1301892)

ELSA-2016-3519 Important Oracle Linux 6 Unbreakable Enterprise kernel security update

Oracle Linux Security Advisory ELSA-2016-3519

The following updated rpms for Oracle Linux 6 have been uploaded to the
Unbreakable Linux Network:







Description of changes:

  • [3.8.13-118.3.2.el6uek]
    • - x86/nmi/64: Use DF to avoid userspace RSP confusing nested NMI detection (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Reorder nested NMI checks (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Improve nested NMI comments (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Switch stacks on userspace NMI entry (Andy Lutomirski) [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/paravirt: Replace the paravirt nop with a bona fide empty function (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}

ELSA-2016-3519 Important: Oracle Linux 7 Unbreakable Enterprise kernel security update

Oracle Linux Security Advisory ELSA-2016-3519

The following updated rpms for Oracle Linux 7 have been uploaded to the Unbreakable Linux Network:



x86_64: kernel-uek-firmware-3.8.13-118.3.2.el7uek.noarch.rpm
x86_64: kernel-uek-doc-3.8.13-118.3.2.el7uek.noarch.rpm
x86_64: kernel-uek-3.8.13-118.3.2.el7uek.x86_64.rpm
x86_64: kernel-uek-devel-3.8.13-118.3.2.el7uek.x86_64.rpm
x86_64: kernel-uek-debug-devel-3.8.13-118.3.2.el7uek.x86_64.rpm
x86_64: kernel-uek-debug-3.8.13-118.3.2.el7uek.x86_64.rpm
x86_64: dtrace-modules-3.8.13-118.3.2.el7uek-0.4.5-3.el7.x86_64.rpm
SRPMS: http://oss.oracle.com/ol7/SRPMS-updates/kernel-uek-3.8.13-118.3.2.el7uek.src.rpm
SRPMS: http://oss.oracle.com/ol7/SRPMS-updates/dtrace-modules-3.8.13-118.3.2.el7uek-0.4.5-3.el7.src.rpm


Description of changes:

  • [3.8.13-118.3.2.el7uek] 
    • - x86/nmi/64: Use DF to avoid userspace RSP confusing nested NMIdetection (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Reorder nested NMI checks (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Improve nested NMI comments (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/nmi/64: Switch stacks on userspace NMI entry (Andy Lutomirski) [Orabug: 22742507]  {CVE-2015-5157}
    • - x86/paravirt: Replace the paravirt nop with a bona fide empty function (Andy Lutomirski)  [Orabug: 22742507]  {CVE-2015-5157}