Bionic Beaver on Z/Architecture: My Personal Mainframe IBM S/390 running Ubuntu 18 Linux on Hercules on OpenSUSE Tumbleweed

Hercules IBM Z/Architecture Mainframe Emulator Console

IBM S/390
Picture courtesy of The Computer Sheds

As a Digital alumnus with reverence for all things DEC, competitor IBM's big iron mainframes and operating systems have always been a curiosity. So far, my IBM experience has been with PC-DOS (here is a PC DOS 2000 based internet-facing web server), their incredible IBM OS/2 Warp, and recently IBM AIX on a virtual box.

Inspired by Astr0baby's blog post and Jeff Sipek's guide, I decided to install Ubuntu 18 "Bionic Beaver" Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-36-generic s390x) on a Hercules-emulated IBM S/390 Mainframe computer running on OpenSUSE Tumbleweed inside a Oracle VirtualBox appliance. This is the first mainframe-class machine emulated at SANYALnet Labs.

Hercules 4.0 Hyperion mainframe emulator was used for the guest S/390.  "Hercules is an open source software implementation of the mainframe System/370 and ESA/390 architectures, in addition to the latest 64-bit z/Architecture." - Hercules official web page.



OpenSUSE Packages

Packages installed in preparation of OpenSUSE Tumbleweed as the build and execution environment for Hercules hypervisor include the following. The standard zypper package management tool for OpenSUSE was used.

# zypper install bridge-utils uml-utilities tunctl net-tools-deprecated ipcalc git cmake vde2 libcap-progs libpcap-devel libpcap1 pcapdump pcapinfo
# zypper install -t pattern devel_C_C++

HOST NETWORK SETUP

OpenSUSE Tumbleweed; IP: 10.100.0.22/24
Guest: S/390, IP Address 10.100.0.23/24
Gateway: 10.100.0.1
DNS: 8.8.8.8 (google DNS)

OpenSUSE's firewall was interfering with the guest S/390s ability to resolve domain names via DNS and access the internet over HTTP(S), both during installation and post-installation of Ubuntu-18 for s390x.  For example, the following message was observed during Ubuntu s390x installation in the guest:

Ubuntu s390x Installer Error accessing Archive Mirrors over Internet

To get around this problem, the host (OpenSUSE) firewall daemon "firewalld" was disabled completely  and a startup script was added to flush and clear iptables rules at boot time.

# systemctl disable firewalld
# systemctl stop firewalld

OpenSUSE host network setup executable script at /root/netsetup/bridge-tap-vde-setup.sh:
-

-

The following was added to /etc/init.d/after.local to get the network setup script to execute at boot time:

#!/bin/bash
#
# --
# /etc/init.d/after.local
# --
touch /forcefsck
/root/netsetup/bridge-tap-vde-setup.sh > /tmp/bridge-tap-vde-setup.sh 2>&1
sync
exit 0

Also the after-local service was enabled for /etc/init.d/after.local script to execute at boot-time:

# systemctl enable after-local.service

# systemctl start after-local.service

IBM S/390 Mainframe in Hercules and Ubuntu 18 (s390x) installation

Hercules startup and CD-ROM boot command

Astr0baby's instructions were followed for rest of the installation of Ubuntu 18 s390x on the guest. The full command to mount the downloaded distribution CD-ROM ISO image on OpenSUSE's /mnt directory is:

$ sudo mount -t iso9660 -o loop ubuntu-18.04.1-server-s390x.iso /mnt

The following hercules.cnf file was used:

-

-

Hercules had to be launched from OpenSUSE root account; even sudo from a user account did not work for letting Hercules access the tun adapter completely for networking. This is despite setting permissions on the /dev/net/tun, /use/local/bin/hercifc etc. as described in "Hercules Version 4: TCP/IP networking with Hercules".

Since any desired Ubuntu packages could be installed later, only the "SSH server" option was selected in addition to the Ubuntu base install in the installation software selection options screen.

The actual Ubuntu 18.04 s390x installation turned out to be uneventful. It followed a similar path to Ubuntu installation on x64. Automatic post-installation reboot did not work as Hercules stopped with halt when the guest operating system shut down for reboot. Exiting and relaunching the emulator and booting the guest operating system worked fine. Here is a video captured during the installation process:

Automatic boot-up of guest Ubuntu s390x on startup of Hercules hypervisor was achieved by creating a file "hercules.rc" in the same directory as "hercules.cnf" with a line containing the same command used at the Hercules prompt to boot manually:

ipl 120

Pressing ESC in the Hercules console screen toggles between a "graphical" view of the S/390 showing processor registers, the processor status word/flags, CPU usage, disk and network I/O etc. as in the example at the top of this post.

Anything typed in starting with a period at the Hercules console's "herc ====>" prompt is sent on to the virtual guest directly (i.e. not processed by the emulator itself). Therefore, even if SSH access to the Ubuntu s390x guest is unavailable, it is possible to login to Ubuntu s390x by entering .username and .password starting with a period (i.e. a dot) at the beginning on the Hercules console, and execute Linux commands by typing them in starting with dots the same way.

It is exciting to be able to run a mainframe version of Ubuntu as a hobbyist system!

-

-

Download

You can download free snippets of the experiment's session logs from my google drive. In addition. here are some random images of screenshots taken during having all this fun!

Pandora FMS and eHorus - a great integrated network monitoring and SaaS cloud-based remote management system

eHorus integration in Pandora FMS web interface (Processes vie)

After playing around with the usual network monitoring tools, all of them impressive (Nagios, PRTG, Zabbix, Zenoss), I have settled down on Pandora FMS for a few years to monitor hobbyist servers in SANYALnet Labs. With solid agent-based real-time performance monitoring and alarming capabilities and an impressive "recon" task with automatic network hierarchy discovery and visual network mapping features, Pandora FMS has been serving me very well.

After a recent upgrade to the latest Pandora FMS distribution, I discovered it supports seamless integration with the eHorus cloud-based remote management system (SaaS) for total command and control of my network nodes right from inside the Padora FMS web interface as well as the eHorus portal internet web-site.

The steps to deploy eHorus and the required registration form and agent downloads are described pretty well at the eHorus web-site. The free tier allows up to 10 nodes and one concurrent user - quite enough for a hobbyist environment like mine.

I started off by registering an account at the eHorus portal and installing the CentOS 7 64-bit eHorus agent on my Dell PowerEdge R710 virtualization host that runs a bunch of SANYALnet Labs hobbyist nodes.

I downloaded and installed the eHorus agent for 64-bit CentOS 7 following these instructions.The only change I made to the /etc/ehorus/ehorus_agent.conf file is to substitute my real eHorus userid in the "#eh_user USER" parameter in the config file.




I then enabled and started the ehorus_agent_daemon using the systemctl command.

# systemctl enable  ehorus_agent_daemon
# systemctl start ehorus_agent_daemon
# systemctl status  ehorus_agent_daemon
● ehorus_agent_daemon.service - LSB: eHorus Agent startup script
   Loaded: loaded (/etc/rc.d/init.d/ehorus_agent_daemon; bad; vendor preset: disabled)
   Active: active (running) since Fri 2018-10-05 23:55:20 UTC; 2h 13min ago
     Docs: man:systemd-sysv-generator(8)
   CGroup: /system.slice/ehorus_agent_daemon.service
           └─20940 /usr/bin/ehorus_agent -f /etc/ehorus/ehorus_agent.conf

Oct 05 23:55:18 dell-poweredge-r710.sanyalnet.lan systemd[1]: Starting LSB: eHorus Agent startup script...
Oct 05 23:55:19 dell-poweredge-r710.sanyalnet.lan ehorus_agent_daemon[20908]: 2018-10-05 23:55:19 [log][2] WARNING: no pas...t!
Oct 05 23:55:20 dell-poweredge-r710.sanyalnet.lan ehorus_agent_daemon[20908]: eHorus Agent is now running with PID 20940
Oct 05 23:55:20 dell-poweredge-r710.sanyalnet.lan systemd[1]: Started LSB: eHorus Agent startup script.
Hint: Some lines were ellipsized, use -l to show in full.

Checking the eHorus web portal, I could now see my server:

eHorus Portal (internet web site) with one server

eHorus provides the following options for command and control of configured servers:

• Terminal
• Desktop,
• Processes
• Services
• Files.

eHorus Node Details Screen at Web Portal

eHorus integrates with Pandora FMS enabling seamless monitoring and control facilities for nodes from right inside the Pandora FMS web UI. Here is an example of a eHorus terminal window inside a Pandora FMS web session:

 eHorus terminal inside Pandora FMS

I will gradually deploy eHorus remote management agents on some of my other nodes. Unfortunately, the eHorus agent is not available for OpenVMS VAX or Alpha, Solaris, AIX, NetBSD and similar unusual operating systems that I play around with.