Search

Wednesday, December 4, 2019

Running my own STRATUM 1 NTP Server

Supratim Sanyal's Blog: Stratum 1 NTP Time Server and GPS location using gpsd on Linux: cgps


I am obsessive about correct clocks, including the ones inside my hobbyist systems and have been running public-service NTP pool servers for many years. However, I have always been limited to implementing Stratum-2 NTP servers so far.

But recently I found this DIYmall VK-162 USB GPS dongle for under thirteen bucks at Amazon, and decided to run my own GPS-synchronized stratum-1 NTP server at home. Here's a link to the item at Amazon (includes my affiliate id):




I tried this on my old trusty Compaq Presario CQ-61 running MX Linux 18.3 Continuum (a Debian derivative). Since the GPS dongle adheres to the National Marine Electronics Association (NMEA) standard protocol, it should work on pretty much any computer with a USB port and NMEA-compliant software that can read serial data from a USB port.

Here's a step-by-step guide on running your own Stratum 1 GPS clock source for your NTP time server.

Plug in the GPS dongle and inspect dmesg. The Communications Device Class (CDC) Abstract Control Model (ACM) driver automatically kicked in and created a serial TTY device "ttyACM0".

$ dmesg
...
...
[27235.388161] usb 3-1: new full-speed USB device number 2 using ohci-pci
[27235.551302] usb 3-1: New USB device found, idVendor=1546, idProduct=01a7, bcdDevice= 1.00
[27235.551311] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[27235.551315] usb 3-1: Product: u-blox 7 - GPS/GNSS Receiver
[27235.551318] usb 3-1: Manufacturer: u-blox AG - www.u-blox.com
[27236.578514] cdc_acm 3-1:1.0: ttyACM0: USB ACM device
[27236.579964] usbcore: registered new interface driver cdc_acm
[27236.579967] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters


"lsusb" shows the USB GPS dongle "U-Blox AG [u-blox 7]" in the list of USB devices:

$ lsusb
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 003: ID 0bda:018a Realtek Semiconductor Corp. 
Bus 001 Device 002: ID 064e:a102 Suyin Corp. Acer/Lenovo Webcam [CN0316]
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 002: ID 1546:01a7 U-Blox AG [u-blox 7]
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

The system log confirms that a device "/dev/ttyACM0" was created. Not only that, the Linux Debian 4.19.37-2 kernel also recognized the GPS ACM device as a Pulse per Second (PPS) clock source.

$ sudo tail /var/log/messages
[sudo] password for localuser: 
Nov 30 22:45:40 compaq-cq61 kernel: [27235.551315] usb 3-1: Product: u-blox 7 - GPS/GNSS Receiver
Nov 30 22:45:40 compaq-cq61 kernel: [27235.551318] usb 3-1: Manufacturer: u-blox AG - www.u-blox.com
Nov 30 22:45:41 compaq-cq61 mtp-probe: checking bus 3, device 2: "/sys/devices/pci0000:00/0000:00:12.0/usb3/3-1"
Nov 30 22:45:41 compaq-cq61 mtp-probe: bus: 3, device: 2 was not an MTP device
Nov 30 22:45:41 compaq-cq61 kernel: [27236.578514] cdc_acm 3-1:1.0: ttyACM0: USB ACM device
Nov 30 22:45:41 compaq-cq61 kernel: [27236.579964] usbcore: registered new interface driver cdc_acm
Nov 30 22:45:41 compaq-cq61 kernel: [27236.579967] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
Nov 30 22:55:50 compaq-cq61 kernel: [27845.239635] pps_ldisc: PPS line discipline registered
Nov 30 22:55:50 compaq-cq61 kernel: [27845.241003] pps pps0: new PPS source acm0
Nov 30 22:55:50 compaq-cq61 kernel: [27845.241034] pps pps0: source "/dev/ttyACM0" added

The "/dev/ttyACM0" device indeed exists now:

$ ls -l /dev/ttyA*
crw-rw---- 1 root dialout 166, 0 Nov 30 22:45 /dev/ttyACM0


We can read the "/dev/ttyACM0" device to see the GPS Dongle reporting data:

$ cat /dev/ttyACM0
$GPTXT,01,01,02,u-blox ag - www.u-blox.com*50

$GPTXT,01,01,02,HW  UBX-G70xx   00070000 FF7FFFFFo*69

$GPTXT,01,01,02,ROM CORE 1.00 (59842) Jun 27 2012 17:43:52*59

$GPTXT,01,01,02,PROTVER 14.00*1E

$GPTXT,01,01,02,ANTSUPERV=AC SD PDoS SR*20

$GPTXT,01,01,02,ANTSTATUS=OK*3B

$GPTXT,01,01,02,LLC FFFFFFFF-FFFFFFFF-FFFFFFFF-FFFFFFFF-FFFFFFFD*2C

$GPRMC,,V,,,,,,,,,,N*53

$GPVTG,,,,,,,,,N*30

$GPGGA,,,,,,0,00,99.99,,,,,,*48

$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30

$GPGLL,,,,,,V,N*64

$GPRMC,,V,,,,,,,,,,N*53

$GPVTG,,,,,,,,,N*30

$GPGGA,,,,,,0,00,99.99,,,,,,*48

$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30

$GPGLL,,,,,,V,N*64

$GPRMC,,V,,,,,,,,,,N*53

$GPVTG,,,,,,,,,N*30

$GPGGA,,,,,,0,00,99.99,,,,,,*48

$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30

$GPGLL,,,,,,V,N*64

^C

So at this point the USB GPS dongle itself is configured and sending data over. Install the gpsd and gpsd-clients packages to read the GPS data and present it to the NTP daemon:

$ sudo apt-get install -y gpsd gpsd-clients
Reading package lists... Done
Building dependency tree     
Reading state information... Done
The following additional packages will be installed:
  libgps22 python-gps
The following NEW packages will be installed:
  gpsd gpsd-clients libgps22 python-gps
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 868 kB of archives.
After this operation, 3,271 kB of additional disk space will be used.
Get:1 http://ftp.us.debian.org/debian stretch/main amd64 libgps22 amd64 3.16-4 [90.2 kB]
Get:2 http://ftp.us.debian.org/debian stretch/main amd64 gpsd amd64 3.16-4 [266 kB]
Get:3 http://ftp.us.debian.org/debian stretch/main amd64 python-gps amd64 3.16-4 [101 kB]                         
Get:4 http://ftp.us.debian.org/debian stretch/main amd64 gpsd-clients amd64 3.16-4 [411 kB]
Fetched 868 kB in 1min 2s (13.9 kB/s)
Selecting previously unselected package libgps22:amd64.
(Reading database ... 267737 files and directories currently installed.)
Preparing to unpack .../libgps22_3.16-4_amd64.deb ...
Unpacking libgps22:amd64 (3.16-4) ...
Selecting previously unselected package gpsd.
Preparing to unpack .../archives/gpsd_3.16-4_amd64.deb ...
Unpacking gpsd (3.16-4) ...
Selecting previously unselected package python-gps.
Preparing to unpack .../python-gps_3.16-4_amd64.deb ...
Unpacking python-gps (3.16-4) ...
Selecting previously unselected package gpsd-clients.
Preparing to unpack .../gpsd-clients_3.16-4_amd64.deb ...
Unpacking gpsd-clients (3.16-4) ...
Setting up libgps22:amd64 (3.16-4) ...
Processing triggers for menu (2.1.47+b1) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
ldconfig: file /usr/lib/x86_64-linux-gnu/libraptor2.so.0 is truncated

ldconfig: file /usr/lib/x86_64-linux-gnu/libraptor2.so.0.0.0 is truncated

Setting up python-gps (3.16-4) ...
Processing triggers for systemd (232-25+deb9u12) ...
Setting up gpsd-clients (3.16-4) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up gpsd (3.16-4) ...
Creating/updating gpsd user account...
Created symlink /etc/systemd/system/sockets.target.wants/gpsd.socket → /lib/systemd/system/gpsd.socket.
Processing triggers for menu (2.1.47+b1) ...
Processing triggers for systemd (232-25+deb9u12) ...

We now modify gpsd daemon's configuration file to use our /dev/ttyACM0 device and to look for satellites as soon as it starts up (the latter tip is from a post by Rapid7):

$ cd /etc/default/
$ sudo cp gpsd gpsd.orig

And edit gpsd using your favorite editor so that it reads:

$ cat gpsd
# Default settings for the gpsd init script and the hotplug wrapper.

# Start the gpsd daemon automatically at boot time
START_DAEMON="true"

# Use USB hotplugging to add new USB devices automatically to the daemon
USBAUTO="true"

# Devices gpsd should collect to at boot time.
# They need to be read/writeable, either by user gpsd or the group dialout.
#DEVICES=""
#Specify the device name:
DEVICES="/dev/ttyACM0"

# Other options you want to pass to gpsd
#GPSD_OPTIONS=""
#-n: look for satellites as soon as it starts up (see https://blog.rapid7.com/2015/07/27/adding-a-gps-time-source-to-ntpd/):
GPSD_OPTIONS="-n"

Here is a diff between the edited and original gpsd configuration file:

$ diff gpsd gpsd.orig
11,13c11
< #DEVICES=""
< #Specify the device name:
< DEVICES="/dev/ttyACM0"
---
> DEVICES=""
16,18c14
< #GPSD_OPTIONS=""
< #-n: look for satellites as soon as it starts up (see https://blog.rapid7.com/2015/07/27/adding-a-gps-time-source-to-ntpd/):
< GPSD_OPTIONS="-n"
---
> GPSD_OPTIONS=""

Now we can restart the gpsd daemon so that the edited configuration is effective:

$ sudo service gpsd restart
[ ok ] Restarting GPS (Global Positioning System) daemon: gpsd.

$ sudo service gpsd status
[ ok ] gpsd is running.

We can now run "cgps" to see the information our GPS dongle is returning (may take a few minutes as the GPS dongle gets fixes on the GPS satellites):

$ cgps
┌───────────────────────────────────────────┐┌─────────────────────────────────┐
│    Time:       2019-12-01T12:40:41.000Z   ││PRN:   Elev:  Azim:  SNR:  Used: │
│    Latitude:    39.191583 N               ││   2    43    102    22      N   │
│    Longitude:   77.234396 W               ││   5    66    033    27      Y   │
│    Altitude:   421.6 ft                   ││   6    03    110    00      N   │
│    Speed:      0.2 mph                    ││   7    04    051    00      N   │
│    Heading:    0.0 deg (true)             ││  13    50    145    28      Y   │
│    Climb:      0.0 ft/min                 ││  15    34    191    31      Y   │
│    Status:     3D FIX (0 secs)            ││  21    10    300    00      N   │
│    Longitude Err:   +/- 123 ft            ││  25    09    240    00      N   │
│    Latitude Err:    +/- 92 ft             ││  26    01    321    00      N   │
│    Altitude Err:    +/- 75 ft             ││  29    58    290    18      N   │
│    Course Err:      n/a                   ││  30    07    081    00      N   │
│    Speed Err:       n/a                   ││                                 │
│    Time offset:     -0.043                ││                                 │
│    Grid Square:     FM19je                ││                                 │
└───────────────────────────────────────────┘└─────────────────────────────────┘
:-77.234396822,"epx":37.684,"epy":28.091,"track":106.8098,"speed":0.102,"eps":1.31}
{"class":"TPV","device":"/dev/ttyACM0","mode":3,"time":"2019-12-01T12:40:41.000Z","ept":0.005,"lat":39.191584000,"lon"
:-77.234396833,"alt":128.500,"epx":37.684,"epy":28.091,"epv":23.000,"track":0.0000,"speed":0.100}
{"class":"TPV","device":"/dev/ttyACM0","mode":3,"time":"2019-12-01T12:40:41.000Z","ept":0.005,"lat":39.191584000,"lon"

{"PRN":6,"el":3,"az":110,"ss":0,"used":false},{"PRN":7,"el":4,"az":51,"ss":0,"used":false},{"PRN":13,"el":50,"az":145,
"ss":28,"used":true},{"PRN":15,"el":34,"az":191,"ss":31,"used":true},{"PRN":21,"el":10,"az":300,"ss":0,"used":false},{
"PRN":25,"el":9,"az":240,"ss":0,"used":false},{"PRN":26,"el":1,"az":321,"ss":0,"used":false},{"PRN":29,"el":58,"az":29
0,"ss":18,"used":false},{"PRN":30,"el":7,"az":81,"ss":0,"used":false}]}
{"class":"TPV","device":"/dev/ttyACM0","mode":3,"time":"2019-12-01T12:40:41.000Z","ept":0.005,"lat":39.191584000,"lon"
:-77.234396833,"alt":128.500,"epx":37.684,"epy":28.091,"epv":23.000,"track":0.0000,"speed":0.100,"climb":0.000}

Press "q" to quite cgps.

The next step is to configure our NTP daemon to use the information from the GSP dongle. Thankfully the gpsd daemon already implements the shared memory area that the NTP daemon understands to use as a clock source.


$ cd /etc
localuser@compaq-cq61:/etc
$ sudo cp ntp.conf ntp.conf.bak
localuser@compaq-cq61:/etc
$ sudo vi ntp.conf

Edit the NTP daemon configuration file to contain the following:

$ cat /etc/ntp.conf
driftfile /var/lib/ntp/ntp.drift

# GPS Serial data reference
server 127.127.28.0 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.0 time1 0.0 refid GPS
# GPS PPS reference
server 127.127.28.1 minpoll 4 maxpoll 4 prefer
fudge 127.127.28.1 refid PPS
# Fallback Stratum 1 Servers
server time.nist.gov
server time.apple.com
server time.google.com

restrict -4 default kod notrap nomodify nopeer noquery limited
restrict -6 default kod notrap nomodify nopeer noquery limited
# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

restrict source notrap nomodify noquery

# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines.  Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient

We restart the NTP daemon to use the modified ntp.conf configuration file:

$ sudo service ntp restart
[ ok ] Restarting GPS (Global Positioning System) daemon: gpsd.

$ sudo service ntp status
[ ok ] NTP server is running.

And voila! After a few minutes, the NTP daemon is a Stratum-1 clock source synchronized to our GPS receiver:

$ ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*SHM(0)          .GPS.            0 l    3   16    7    0.000    4.628   0.967
 SHM(1)          .PPS.            0 l    -   16    0    0.000    0.000   0.000
 time-e-b.nist.g .NIST.           1 u  208   64   10  152.351  114.389   0.000
+usqas2-ntp-001. .SHM.            1 u   10   64   17    6.130   55.096   4.959
+time4.google.co .GOOG.           1 u   12   64   17   13.620   56.659   7.378


$ ntpdate -q localhost
server ::1, stratum 1, offset -0.000023, delay 0.02576
server 127.0.0.1, stratum 1, offset -0.000002, delay 0.02571
 1 Dec 08:03:37 ntpdate[7845]: adjust time server 127.0.0.1 offset -0.000002 sec


Tuesday, November 26, 2019

Windows SSD Solid-State Drive Maximum Life Optimization Summary

Summarized actions to optimally use SSD storage drive with Windows 7 and Windows 10 for the longest possible life of the SSD.



  • Disable automatic page file management and manually allocate a fixed-sized page file whose size is double the amount of RAM on the computer
  • Disable the "Windows Search" search indexer service
  • Disable the "SysMain" service (this was called the "Superfetch" service in earlier versions of Windows before Microsoft changed the name to "SysMain")
  • Delete the search index file C:\ProgramData\Microsoft\Search\Data\Applications\Windows\Windows.edb
  • Turn off hibernation: from a privileged command prompt, execute the command "powercfg.exe /h off"
  • Make sure the TRIM function of SSD ATA interface is enabled. From a privileged command prompt, enter the command "fsutil behavior query DisableDeleteNotify". If the result is not 0,  force the behavior to 0 using "fsutil behavior set DisableDeleteNotify 0".

That's it. Your SSD will last longer now.

Saturday, November 16, 2019

Powering up a British ZX Spectrum after 30 years in America - The PAL UHF Analog TV and 220V AC Challenge


Supratim Sanyal's Blog: Sinclair ZX Spectrum +3 Running in USA with Conversion of 110V to 220V AC and PAL UHF TV swapped with RGB to SCART and SCART to HDMI on NTSC / ATSC US TV Standard


My dear loving world-traveling father brought home a first-generation Sinclair ZX Spectrum (the basic 16K model) in 1983. It is hard to say if he or I got instantly addicted more to the fascinating world of home computing using our first microcomputer.

Eventually the machine was upgraded to 48K of RAM, and then further to ZX Spectrum+ when Sinclair released that upgrade also in kit form with a better keyboard and an awesome new Reset button with instructions on wiring it. I still have that machine, but it does not power up any more.

Fortunately, when I got a bit older, I had obtained a ZX Spectrum +3. It was quite a jump from the original Spectrum with Amstrad's additions of a 3-inch CF2 floppy drive, enhanced ROM, real sound and 128K of RAM. This machine was used heavily all the way to 1988 when I put it back in the box, taped it up and basically hoarded it along with a joystick and Multiface-3 interface.

Three decades would pass while I moved to Japan and then to the United States, got married, had two children checking their pre-K through High School and college boxes and in general living the quintessential nondescript suburban American life. The Spectrums waited patiently in their taped-up boxes.

I started to really want to play with the ZX Spectrum again a couple of years ago and commenced on searching for options for connecting a microcomputer whose power adapter expects 220V 50 Hz AC mains and emits video in analog British PAL UHF television system Channel 35.

Supratim Sanyal's Blog: 110V AC to 220 V AC Step Up Transformer for running ZX Spectrum from 110V AC Mains Power
The AC mains voltage conversion problem was easily solved: there are numerous 110V to 220V step up (and conversely step down) AC transformers available at Amazon.

I never considered the 10 Hz difference in AC mains frequency of any major consequence. The worst that would happen is something like the 3-byte FRAMES counter at address 23672 increment slightly faster than every 20 milliseconds, wouldn't it?


I ended up getting the TriGear TG100, which works great and keeps the ZX Spectrum +3 power brick happy.


Here is a picture of the Trigear TG-100 110v-220v AC transformer feeding 220V AC to the ZX Spectrum +3 Power adapter from our US standard 110V AC mains line:

Supratim Sanyal's Blog: 110V AC to 220 V AC Step Up Transformer for running ZX Spectrum from 110V AC Mains Power
TriGear Step Up / Down Transformer to supply 220V AC to Sinclair ZX Spectrum Power Adapter


Displaying the ZX Spectrum +3 PAL UHF video signal on a modern NTSC/ATSC TV (or computer monitor) proved to be more challenging. I unsuccessfully tried two different approaches before the third approach worked. I will start off by first describing the two failed attempts and why they failed.


Failed attempt #1: PAL TV Tuner to VGA converter box


I purchased one of these PAL/NTSC LCD Analog TV Tuner Box PIP Computer TV Program Receiver units from eBay. The unit is great - it comes with an old British-style female antenna jack that the ZX Spectrum's video cable plugs straight into. It converts PAL antenna signals (and all sorts of other video input signals) into standard VGA output, which then can be connected to a regular computer monitor or television that has a VGA port. It comes with a remote controller, power supply, a cable to connect the DIN output to a VGA monitor, and on-board software that sets it up including a channel scanning feature.

The problem with this unit is there is no user-accessible fine-tuning control. The ZX Spectrum's video modulator circuit is analog and not totally accurate, tending to drift. My attempts to get the video working with this unit were not successful; the closest I got was an extremely faint ZX Spectrum +3 main menu barely visible in the raster noise at channels 33 and 34. There is no opportunity to fine-tune from there.

Failed Attempt #2: PAL/NTSC Digital Analog Hybrid TV Tuner on a Windows XP PC


I found this PAL/NTSC Digital/Analog Hybrid TV tuner card on Amazon and installed it in the only PCI-Express (PCI-e) slot on a old desktop computer running Windows XP. It comes with a mini-CD with the driver and software to watch TV as well as a little adapter dongle that connects its female UK-style PAL antenna connector and presents a female US-style coaxial antenna connector. The ZX Spectrum video cable connects to it without the adapter, of course.

The attempt at getting video from the  ZX Spectrum displayed on the Windows XP computer's screen failed due to exactly the same reason as the PAL-to-VGA converter box: there is no way to manually fine-tune the tuner and the ZX Spectrum does not emit a precisely modulated video signal at the promised PAL UHF channel 35 frequency any more. Once again, I could see the faint outline of the ZX Spectrum +3's main menu in the grainy raster of channels 33 and 34, but had no way to fine-tune into the drifted analog video signal.

Finally, Success!

Supratim Sanyal's Blog: Sinclair ZX Spectrum +3 Running in USA with Conversion of 110V to 220V AC and PAL UHF TV swapped with RGB to SCART and SCART to HDMI on NTSC / ATSC US TV Standard


With little hope of finding an analog PAL UHF tuner with manual fine-tuning, I gave up on trying to tune into ZX Spectrum +3's PAL UHF antenna signal and focused instead on it's RGB/Peritel output. The RGB/Perital port had never been used before, but it was time to put it to work. Peritel is another name for SCART, and is also popularly known as the Euro Connector.

First, I found and ordered a RGB to SCART Cable. They are apparently still being manufactured and available brand new from multiple sellers in the United Kingdom. I got mine from Coolnovelties Retrogaming, Electronics and Novelty Gifts online store.

But, the TV I wanted to hook the Spectrum +3 to does not have a SCART port. It does have multiple HDMI ports. So now, I also needed something to convert the A/V signal from a SCART connector to HDMI. Luckily I found this TNP SCART to HDMI Converter Video Audio Adapter Box with SCART/HD Switch, PAL/NTSC Video Scaler, 1080P/720P Upscaler Support HDMI Connector Output, 3.5mm AUX Jack and Coaxial Audio Output which does the job perfectly.

Supratim Sanyal's Blog: SCART to HDMI converter for ZX Spectrum +3Supratim Sanyal's Blog: SCART to HDMI converter for ZX Spectrum +3



That was it! The two-step conversion of the ZX Spectrum +3 RGB signal to SCART and then to HDMI works great. Here are a couple of pictures of the working setup.

Supratim Sanyal's Blog: Sinclair ZX Spectrum RGB to SCART Video Conversion
Sinclair ZX Spectrum +3 RGB to SCART Video Converter

Supratim Sanyal's Blog: Sinclair ZX Spectrum SCART to HDMI Video Converter Input
Sinclair ZX Spectrum +3 SCART to HDMI Video Converter: Input Side

Supratim Sanyal's Blog: Sinclair ZX Spectrum +3 SCART to HDMI Video Converter: HDMI Output
Sinclair ZX Spectrum +3 SCART to HDMI Video Converter: HDMI Output

Finally, if you are wondering how I am loading Jetpac from cassette tape in the picture at the beginning of this post, I am using a Raspberry Pi to feed audio from a TZX file to the ZX Spectrum +3; this will be the subject of my next post.

Please leave comments about your thoughts, or alternative ways you may have achieved a functional ZX Spectrum in a US 110V AC voltagae and NTSC/ATSC TV environment.


Monday, November 11, 2019

Windows 10: Move Partitions for Contiguous Unallocated Space to Extend Existing Partitions

The problem:

Windows 10 Partitions with unused free space: Windows Disk Management Screen


An old laptop (HP 2000) running Windows 10 Pro was upgraded from 80 GB 2.5" SATA Hard Disk Drive to 240 GB 2.5" SATA Solid State Drive (SSD). I used Clonezilla to back up the old 80 GB drive and restore to the new larger SSD. This worked fine as expected. However, I am left with a totally unused 149 GB space on the new SSD.

I cannot extend the Windows 10 partition into the unused space using Windows Disk Management administration tool because the recovery partition is right next to the Windows partition and before the empty space preventing a merge of the unallocated unused space with the Windows partition.

So I have to find a way to move the recovery partition to the end of the disk so that the empty space is moved left, next to the Windows partition. Then I can extended the Windows partition to use the unallocated space.

The solution:

The top search result: EaseUS Partition Master. There is a free edition for personal / home use. The downloaded installer filename is "epm_free_installer.exe". After a quick Virustotal scan, I proceeded to install it. The installer was a stub which downloaded the real 33.44 MB installer and completed the installation fine after taking significant time to get beyond the 93% completion mark. On launching it and approving a UAC check, a pretty neat initial information screen was presented:

EaseUS Partition Manager initial screen


The program queues up the desired operations and executes them when all operations are in the queue. It does not make any immediate changes as the list of operations is constructed.

The first operation: Right-clicking on the third partition produced a "Resize/Move" menu option. Choosing that option resulted in the following screen in which I simply changed the "Unallocated After" value to 0, moving the partition to the end of the disk:

EaseUS - moving partition to end of disk

The second operation: creating a partition in the free space. Again, right-clicking on the 3rd partition (now occupied by the unallocated empty space) produced a menu with a "Create" option to create a partition in the unallocated space:

Windows 10 NTFS - EaseUS Create Partition in Unallocated Space

Windows 10 NTFS Partition Management - Create new partition in empty space


The third operation: Now that we have the empty partition next to the existing Windows partition, we can merge the two to extend the Windows partition into the empty space. Again, right-clicking on the 3rd (empty) partition and choosing "Merge" produces check boxes in the partitions to select the partitions to merge, which in this case is the 3rd partition as the source of the merge and the existing Windows (2nd) partition as the target of the merge. Note: Only empty partitions can be merged into existing Windows boot partition but not the other way around; EaseUS conveniently warns if this rule is not followed, and does not enable the OK button.

Windows 10 NTFS Merge Extend Boot Partition with Empty Partition


This is what I wanted. At this stage, EaseUS displays a play button at the top left of the screen labeled "Execute 3 Operations". On clicking the Play button, it pops up a "Pending Operations" screen summarizing what is to be done with a warning of a reboot being required along with Apply and Cancel buttons. On Applying the changes and rebooting, I had the desired results:

Windows 10 Merged Boot System Primary Partition

Happy!


Saturday, November 9, 2019

HPUX 11i v1 (HPUX 11.11) PA-RISC Guest Virtualization using QEMU on FreeBSD Host

Supratim Sanyal's Blog: Virtual HPUX 11i CDE Desktop on QEMU for FreeBSD


HP-UX is one of the last great commercial-grade Unix derivatives of the legendary Unix System V, and is a mandatory installation for all retro-computing and Unix enthusiasts.

Unix history-simple.svg
By Eraserhead1, Infinity0, Sav_vas - Levenez Unix History Diagram, Information on the history of IBM's AIX on ibm.com, CC BY-SA 3.0, Link


I jumped on Astr0baby's recently published "Running HP-UX 11.11 on qemu-system-hppa" about running HP's solid enterprise-class HPUX Unix version 11i v1 (HPUX 11.11) on the venerable PARISC processor now emulated completely by QEMU. Proceeding to download the latest QEMU development source from github and building qemu-system-hppa on my FreeBSD Release-12 host running on a Dell PowerEdge 2950, and downloading the HPUX 11i ISO CD images, I fired up the QEMU virtual machine and soon had HP-UX Version B.11.11 running on a HP 9000/778/B160L.

Supratim Sanyal's Blog: HPUX 11i running in QEMU qemu-system-hppa on FreeBSD virtualization host


Remarkably, almost as soon as I had HPUX 11.11 running, the maintainer of QEMU HPPA emulator checked in new code that supports Digital Equipment Corporation's Tulip network card, as pointed out almost immediately by one of the folks following Astr0baby's post. A fresh pull and rebuild of the QEMU source enabled networking on the HPUX virtual machine, which also means the nostalgic HPUX CDE desktop environment is now accessible over network as in the screenshot above.

I prefer to use VDE (Virtual Distributed Ethernet) for my virtualization experiments. QEMU has great support for VDE networking as does Oracle VirtualBox. However, attempting to build QEMU with VDE configured on FreeBSD requires the VDE development library on FreeBSD which I have not located yet. Network connectivity is for now using standard TAP interfaces bridged into a VDE switch. The script that sets the VDE switch and bridges TAP interfaces into it is documented in my prior post here.

The script that launches the HPUX virtual machine looks like this:

#!/bin/sh

# Launches Guest:
# 10.42.2.55 hpux.sanyalnet.lan hpux

# Change -boot to d to boot from cdrom or c to boot from hdd

# Careful of MAC address conflicts in network! Last HEX pair of MAC = last byte of guest IP.
# Also, VNC screen, if any, on dispaly number = last byte of guest IP
# And Console TELNET port = 20800 + last byte of guest IP

TELNET_CONSOLE_PORT=20855
screen -m -d -S HPUX-QEMUMON \
./qemu-system-hppa \
    -boot c \
    -m 512 \
    -drive if=scsi,bus=0,index=0,file=./hpux-striped-8gb-disk-1.qcow2.dsk,format=qcow2 \
    -drive if=scsi,bus=0,index=1,file=./hpux-striped-8gb-disk-2.qcow2.dsk,format=qcow2 \
    -cdrom ./ISO/hpux11-01.iso \
    -net nic,model=tulip,macaddr="58:9C:FC:52:54:37" \
    -net tap,ifname=tap4,script=no \
    -serial telnet::${TELNET_CONSOLE_PORT},server \
    -serial mon:stdio \
    -nographic \
    -d nochain \
    -vnc :55 \
    ####-smp cpus=2 ## -accel tcg ###,thread=multi

sleep 2

screen -m -d -S HPUX-CONSOLE telnet localhost ${TELNET_CONSOLE_PORT}

echo At this point, qemu should be on screen HPUX-QEMUMON and console telnet session on screen HPUX-CONSOLE
echo You can launch a CDE Graphical Desktop Session using Xephyr like the following:
echo Xephyr -screen 1200x720 -ac -query 10.42.2.55 :55

sync;sync;sync
exit 0

For the HPUX/PARISC virtual machine, I configured two 8GB virtual disks and told HPUX to use the two disks as a striped logical volume group. The two virtual disks are in QEMU's QCOW v2 (acutally QCOW v3) format. Similar to RAID0, using disk striping promises to significantly improve performance, at the cost of providing 8GB total capacity despite the two 8GB disks.

$ ls -l *dsk
-rw-r--r--  1 localuser  localuser  622919680 Nov  9 17:30 hpux-striped-8gb-disk-1.qcow2.dsk
-rw-r--r--  1 localuser  localuser  550502400 Nov  9 17:30 hpux-striped-8gb-disk-2.qcow2.dsk

$ file *dsk
hpux-striped-8gb-disk-1.qcow2.dsk: QEMU QCOW Image (v3), 8589934592 bytes
hpux-striped-8gb-disk-2.qcow2.dsk: QEMU QCOW Image (v3), 8589934592 bytes

I did not see any major performance improvements with QEMU's accelerator or with using two virtual CPUs to enable SMP (-smp cpus=2 -accel tcg,thread=multi) though HPUX had no problem recognizing the additional CPU. In fact, performance may have been worse than using just one CPU with no acceleration, so I disabled acceleration and multiprocessor support in QEMU. Here is the report from ioscan:

Class       I  H/W Path     Driver    S/W State   H/W Type     Description
===========================================================================
bc          0               root      CLAIMED     BUS_NEXUS    
bc          1  8            bc        CLAIMED     BUS_NEXUS    Pseudo Bus Converter
ba          0  8/0          GSCtoPCI  CLAIMED     BUS_NEXUS    GSCtoPCI Bridge
ext_bus     0  8/0/0/0      c720      CLAIMED     INTERFACE    SCSI C895 Fast Wide LVD
target      0  8/0/0/0.0    tgt       CLAIMED     DEVICE       
disk        0  8/0/0/0.0.0  sdisk     CLAIMED     DEVICE       QEMU    QEMU HARDDISK
                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
target      1  8/0/0/0.1    tgt       CLAIMED     DEVICE       
disk        1  8/0/0/0.1.0  sdisk     CLAIMED     DEVICE       QEMU    QEMU HARDDISK
                           /dev/dsk/c0t1d0   /dev/rdsk/c0t1d0
target      2  8/0/0/0.2    tgt       CLAIMED     DEVICE       
disk        2  8/0/0/0.2.0  sdisk     CLAIMED     DEVICE       QEMU    QEMU CD-ROM
                           /dev/dsk/c0t2d0   /dev/rdsk/c0t2d0
target      3  8/0/0/0.7    tgt       CLAIMED     DEVICE       
ctl         0  8/0/0/0.7.0  sctl      CLAIMED     DEVICE       Initiator
                           /dev/rscsi/c0t7d0
lan         0  8/0/1/0      btlan     CLAIMED     INTERFACE    HP PCI 10/100Base-TX Core
                           /dev/diag/lan0  /dev/ether0     /dev/lan0     
tty         0  8/0/63       asio0     CLAIMED     INTERFACE    Built-in RS-232C
                           /dev/diag/mux0  /dev/mux0       /dev/tty0p0   
processor   0  48           processor CLAIMED     PROCESSOR    Processor
memory      0  63           memory    CLAIMED     MEMORY       Memory

HP-UX CDE Graphical Desktop


Initially, I was having trouble accessing the HPUX CDE graphical desktop using the usual XDMCP client (Xephyr) over network. An hourglass would show up, then disappear, and show up again, as if the X session was continually trying to start up and fail. Eventually I realized a core file was being created by "dtgreet" and traced the problem to "dtgreet fails and dumps core if the default font is unavailable for the selected locale". Although a patch is apparently available to address this, I wanted to fix the underlying issue of the missing default font. Further online research led to "Configuring HP-UX for Different Languages". Surely enough, the /etc/dt/config/Xconfig file had Dtlogin*language set to <lang>; simply editing the file to say "Dtlogin*language: en_US.iso88591" followed by running /sbin/init.d/dtlogin.rc reset immediately got HP-UX CDE working from the FreeBSD host machine:

$ exec Xephyr -screen 1200x720 -ac -query 10.42.2.55 :55

Supratim Sanyal's Blog: HPUX CDE Graphical Environment Start Up


Installing Software

I registered at HPE Software Depot describing myself as a Hobbyist and am able to download software via "Electronic Delivery". The software comes in the form of files with long filenames and  .depot extension, and the filenames have to be preserved for the swinstall tool to install them.

There was an initial hiccup in getting the swinstall tool to install the downloaded software: I was getting the error:

You do not have required permissions to perform this operation. Check permissions using the "swacl" command.

After attempting a lot of things mentioned online, the steps mentioned here regarding setting the acls worked:

1) Save the current (insufficient) host acl to a file:
 # swacl -l host > swacl.host

2) Edit swacl.host and givc all permissions ("a") to the usernames. After this change, swacl.host looks like:

user:root:a
user:localuser:a
group:swadm:a
any_other:-r---

3) Enable the modified acl:

# swacl -l host -F swacl.host

4) Restart the software agent daemon:

# /sbin/init.d/swagentd stop
# /sbin/init.d/swagentd start

Mozilla Web Browser on HP-UX 11i

Supratim Sanyal's Blog: Mozilla Web Browser on HP-UX 11i CDE Graphical Desktop


I downloaded PA_RISC_Firefox_3.5.09.00_May_10_ffox_A3509ar_pa.depot from the HPE Software Depot and transferred it to HP-UX's /var/spool/sw/ directory using FTP. The HP-UX swinstall tool was then invoked successfully to install Firefox.

# swinstall -s /var/spool/sw/PA_RISC_Firefox_3.5.09.00_May_10_ffox_A3509ar_pa.depot

Supratim Sanyal's Blog: Install Mozilla Firefox Web Browser on HPUX 11.11 PARISC

Supratim Sanyal's Blog: Install Mozilla Firefox Web Browser on HP-UX 11.i v1 PARISC


Unfortunately, the downloaded Firefox does not run on the PA7300LC processor of the emulated platform (it needs PA-RISC 2.0):
$ pwd
/opt/firefox
$ ./firefox-bin
sh: ./firefox-bin: Execute permission denied.
$ file firefox-bin
firefox-bin: PA-RISC2.0 shared executable dynamically linked -not stripped



Fortunately, a working Mozilla web browser was installed along with HP-UX 11.11, and can be invoked by typing in "mozilla" at a command prompt. Java support is questionable since Mozilla's text console (stdio) launch output indicates the Java plugin for PARISC 2.0 is not of the right executable format for the emulated PARISC 1.1 CPU.
$ pwd
/opt/mozilla
$ file mozilla-bin
mozilla-bin: PA-RISC1.1 shared executable dynamically linked -not stripped



HP even email back a very corporate-looking receipt for the electronic software distribution!

Supratim Sanyal's Blog: Mozilla Firefox for HP-UX 11i - Hewlett Packard Enterprise Electronic Delivery Receipt



Next, I grabbed depothelper from the awesome HP-UX Porting and Archive Centre and tried to install a couple of packages. With a one-line tweak of adding a command to FTP for Passive Mode (depothelper is just a script), the tool does an excellent job of pulling dependencies needed for the packages. However, my attempt to grab "wget" failed with it unable to download the first dependency. "bash" did slightly better in successfully downloading and installing all dependencies, but again wouldn't run on the QEMU's PA7300LC PA-RISC 1.1 processor because it expects PA-RISC 2.0.

The default HP-UX B.11.11 installation does come with a functional C compiler. Here is aclock compiled and running in a dtterm.

Supratim Sanyal's Blog: ACLOCK ASCII CLOCK ON HP-UX 11i (11.11) on HP PARISC


Installation Logs

I mostly remembered to captured the terminal sessions while performing the installation of HP-UX 11.11. Here are a couple for reference.

Initial Install - Virtual HP 9000/778 Console Log:






Initial Install - QEMU Monitor Log:






Post-Install First Boot - Virtual HP 9000/778 Console Log:








Recommended Products from Amazon