USB Drive Interface for the HP/IDE Disk Controller

(still somewhat under construction but getting there)

This page describes connecting a FTDI/Vinculum VDRIVE2 module to the 8052-based HP/IDE disk controller design I have connected to my HP2113E minicomputer for saving and restoring a SimH HP2100-compatible disk image to and from a USB thumbdrive. Previously I was using a USB drive tray for this purpose but that has prevented being able to put the thing in a real box, isn't good on the cables and drive, and required using disk imaging software with byte-swap capability. The USB solution is slow (about 6KB per second) but can be operated in the controller's "debug mode" without having to run the minicomputer, so its slowness is usually not an issue. The HP-IPL/OS system I use has both IDE and 7906 disk drivers, permitting direct copy of the system to and from a 7906 disk image which can be used under simulation. This is essentially the same system in the 7906 disk simulation package, which can be copied as-is to the IDE drive using the restore feature. The mod is also useful for backing up the disk of a pure-IDE SFS-based HP-IPL/OS system even if not simulating. In addition to a backup/restore solution, the new controller code also permits directly accessing the VDRIVE2 and USB files from the minicomputer.

The toggle switch is for selecting the operating mode, to the left is "debug mode" where I can save and restore IDE disk images to and from a USB drive using a serial terminal to make selections, in the middle it just boots to Paulmon, to the right automatically starts the interface code for normal operation, optionally sending a "BOOT" file from the USB drive to the HP first. I still probably should add a push-button switch for resetting the controller next to the toggle switch, after booting the HP, the reset button on the PCB has to be pressed to resume being a disk interface.


The VDRIVE2 and toggle switch was mounted on the level-shifter board using hot-glue, added an extra 10K pullup resistor for the PF2 line used to engage auto-mode by switching low. Also added 470 ohm resistors to the PF0 (debug) and PF2 lines in case they get selected outputs, optional but probably not a bad idea. The added 1M resistor on the right side of the board weakly pulls PF3 low should the VDRIVE2 becomes disconnected, causing 0 to be read rather than random data, optional (leftover from debugging).

Since the photos were taken I added additional power supply decoupling to both +5V/GND proto area connections, including to where the VDRIVE2 is powered from.

Connecting the VDRIVE2

The VDRIVE2 module is connected to the IDE interface circuit as follows...

Pin 1 (ground) to Ground
Pin 2 (data out) to Port F bit 3
Pin 3 (power) to +5V
Pin 4 (data in) to Port F bit 4
Pin 5 (clock) to Port F bit 5
Pin 6 (select) to Port F bit 6

Port F is Port C of the 2nd 8255 chip. To make use of the optional "Auto" program then a 10K pullup should be connected from +5V to Port F bit 2, with a switch pulling the line low to automatically start the interface code without having to make the interface code itself autostarting.

If using the level-shifters depicted in the IDE interface schematic then a 150 ohm 1/2W resistor load needs to be connected between 5V and GND to bleed off leakage from the HP. My proto already had a 330 ohm 1/4W for ensuring the 8052 resets but that didn't drag the voltage low enough for the VDRIVE2, added another 270 ohm 1/4W (total about 150 ohms) to reduce the supply leakage to about 1.2V for reliable reset. Originally I measured this at 0.6V but it depends on the data state, 100 ohm 1/2W might be a better load.

Additional power supply decoupling should be added at the point where the VDRIVE2 is powered from. I used a 47uF capacitor in parallel with a 0.1uF capacitor for the higher frequencies. The VDRIVE2 maintains important data structures in memory, so a glitch or excessive noise on the power supply lines might cause unpredictable (and unrepeatable) behavior.

The firmware in the VDRIVE2 module had to be upgraded to work properly! Upgrading is easy, just download the latest VDAP ".ftl" firmware from the Vinculum website (I'm using version 3.66), put it on the thumbdrive and rename to "FTRFB.FTL", and insert into the powered VDRIVE2 module. Use the FWV command at the debug mode's "dos prompt" to verify the upgrade. Also note that the firmware manual versions do not correspond to firmware code versions, manual version 2.05 is for firmware version 3.66, and the 2.08 firmware that came in my unit was in fact quite old.

The thumbdrive or USB hard drive must contain a FAT-based file system with 512 byte sectors in the first partition. FAT32 is supported but long filenames are ignored and can cause file naming issues - if a file is renamed a PC continues to see the original name despite the name change. Avoid using the VDRIVE2 to rename files copied from a PC. After writing files to a stock (multiple FAT) format, the Windows XP disk check function reports no errors but the Linux fsck.msdos utility complains about the FATs not matching. To avoid this the device can be formatted with a single FAT using the mkfs.msdos utility with the -f 1 option.

Controller Code

The assembly sources are for the AS31 assembler, configured for the Paul Rev5 dev board. The main source is configured to not autostart and includes an init program at the end to select 9600 baud (@22mhz), remove this part if using a Paulmon with a hard-coded baud rate. I use the "Auto PTR" program to selectively boot the HP and afterwards run the disk interface rather than making the interface code autostarting, thus preserving access to Paulmon for debugging and running other programs.

This version of the HP/IDE disk controller code adds disk image save and restore options to the existing serial-access debug menu, and adds an additional menu for sending and receiving bytes to and from the VDRIVE2 module and interacting with the USB file system from a "dos" prompt. Care must be taken when using these functions, the send/receive options are intended for testing and exactly determining the protocols of various commands, it's quite easy to lock up the VDRIVE2 module if incorrect sequences are given (but that's the point, to find out what works and what doesn't). The prompt can be used maintenance and general browsing of the USB file system, but is not intended for manually opening and writing files. In particular, each single-line command must return output or it will hang, reset to recover. Some error states require cycling the power to recover, and if experimenting keep a disk checker/fixer handy. The "light" version does not include the extra debug functions except for the prompt for file maintenance.

The image save option prompts to save a 7906-sized platter (about 9.6 megabytes) or to save from one to nine 64-file SFS volumes (about 4.1 megabytes for the first volume and 4 megabytes for each additional volume). The image restore option restores however many complete 1KW blocks are present in the disk image file. The bytes in each 16-bit word are swapped on save and restore for SimH compatibility. These operations require about 3 minutes for each megabyte of data saved or restored, save operations must complete to avoid "lost cluster" file system errors.

The controller code provides additional HP/IDE interface commands to send and receive SPI bytes, call the sync and response clear functions, and functions for unstructured reading and writing buffered "stream" files. The VDOS words permit running a USB dos prompt from HP-IPL/OS, streaming read and write files, and sending raw Vinculum commands for writing file-processing programs - however expect lockups and USB file system errors when debugging raw dos code! The streaming functions are much easier to use, these permit assigning read and/or write files to HP-IPL/OS' existing "MS" (mass storage) system which works much like papertape. Files are only actually open when filling or empying a buffer, creating the illusion of both read and write files being "open" at once and protecting the file system should the write close command not be given. Write files must be closed to write the remaining buffer data and avoid an incomplete file, if forgotten and the thumbdrive is removed it can be reinserted and the close command given, provided no other stream operations have been performed.

Adventures in SPI-land

The first task for using the VDRIVE2 module is establishing a SPI connection, this requires the microcontroller to be able to flip bits on three output lines and read one input line without glitches. Some processors have a built-in SPI port, I don't know much about these but to be useful it must be able to read the SPI port without blocking and return a status bit which the VDRIVE2 sets to 1 if it is busy or there is no more data. For other processors the SPI protocol must be "bit-banged" by individually flipping output bits and reading the input bit. Beware of bitset/bitclear instructions which sometimes cause brief upsets on the other pins, rather a variable should be used where one bit is set or cleared at a time and copied to the output port register. For this project the only available bits were on port C of the 2nd 8255 I/O chip (labeled port F on the Paul dev board), good thing since only port C/F can be split into both input and output pins. Don't expect great speed bit-banging SPI on a 8255, I'm getting roughly 8KB a second.

Here's the algorithm I'm using to send and receive SPI bytes (more or less)... [10/18/08]

variables:   data_register, count
output bits: clock, select, data_out
input bit: data_in
flag: status

set up port control bits and variables to access the SPI device
clear clock, select and data_out bits

clear select and data_out bits (clock bit should be clear on entry)
set clock bit
clear clock bit
set clock bit
clear clock bit
set select and data_out bits
  set clock bit ;send 1
clear clock bit
set clock bit ;send another 1 for read
 clear clock bit
clear data_out bit
set clock bit ;send 0
clear clock bit
data_register = 0
count = 8
rloop: shift data_register left
data_register bit 0 = data in bit
set clock bit
clear clock bit
decrement count
if count <> 0 then goto rloop
status = data_in bit
set clock bit
clear clock bit
clear select and data_out bits
set clock bit
clear clock bit

clear select and data_out bits (clock bit should be clear on entry)
set clock bit
clear clock bit
set clock bit
clear clock bit
set select and data_out bits
set clock bit ;send 1
clear clock bit
clear data_out bit
set clock bit ;send 0 for write
clear clock bit
set clock bit ;send another 0
clear clock bit
count = 8
wloop: rotate data_register left into carry flag
data_out bit = carry flag
set clock bit
clear clock bit
decrement count
if count <> 0 then goto wloop
status = data_in bit
set clock bit
clear clock bit
clear select and data_out bits
set clock bit
clear clock bit

call ReadSPIbyte
if status flag set then goto ReadSPIwait

call WriteSPIbyte
if status flag set then goto WriteSPIwait

Note: don't consider this algorithm "correct", it's just what (I think) my code does. In my code InitSPI also sets the DPTR address pointer to the port address and clears an output "shadow" register in which to flip bits to copy to the port as one step. InitSPI must be called before using the SPI read/write functions after printing, buffer operations or anything else that might disturb DPTR. The data_register is kept in R4 and status is returned as a non-zero value in the accumulator.

Programming the VDRIVE2 isn't exactly easy, in SPI mode it's hard to determine if a invalid response means end-of-data or if it just means it's thinking about what to send next. To counter this it's necessary to syncronize the unit so further responses correspond with the last command that was sent, to do this I'm currently using the following algorithm... [updated 10/10/08]

sync:   call the clear code
        send E[cr] to SPI (wait to be accepted)
        read byte from SPI (wait for response)
        if byte isn't 'E' goto sync
        read byte from SPI (wait for response)
        if byte isn't 13 goto sync
        send e[cr] to SPI (wait to be accepted)
        read byte from SPI (wait for response)
        if byte isn't 'e' goto sync
        read byte from SPI (wait for response)
        if byte isn't 13 goto sync
        call the clear code

The clear algorithm is:

clear:  initialize SPI interface
clear1: read byte from SPI (don't wait)
        if valid goto clear1
        count = 100
cloop:  delay a bit
        read byte from SPI (don't wait)
        if valid goto clear1
        decrement count
        if count not 0 then goto cloop

"Delay a bit" in my code is 200 djnz instructions in a loop. The sync code is called to establish communications with the VDRIVE2 and ensure upon return no output is pending. The initialize SPI interface subroutine sets up the port and dptr to permit using the read byte and write byte subroutines, which have extra wrapper subs which read until valid, and write until accepted. The clear subroutine is called by sync to remove responses from the output stream, and also called when the rest of an immediate response needs to be discarded, in particular after error responses or after sending commands and I don't care what it says in response (switching to short commands, binary values, etc). The clear code waits for a bit to see if a response is coming but does not wait if a delayed response is coming, so won't clear responses after file opens or other commands which require processing time.

Presenting a "dos prompt" in which Vinculum commands can be typed and responses displayed is very handy for listing and changing directories, deleting and renaming files, listing the contents of files and other tasks. The prompt code must send "ECS" and "IPA" commands when starting to ensure plain text commands and responses. Don't attempt to use a prompt to directly write files, the exactness of the protocol almost guarantees locking it up and losing a cluster in the process and the VDRIVE2 does not issue a response until all bytes specified in the write command have been received. The 8052 prompt code depends on a response being sent after every command and there's no provision for editing a command line. The prompt implemented in the VDOS package for HP-IPL/OS is a little kinder, the 8052 interface code implementing the SPI byte read waits a little bit but if no response after a timeout period returns with an invalid bit set and it simply prints nothing, and backspace works for correcting mistakes before sending the command. But still don't attempt to directly write files from a prompt. One little problem when displaying responses - the Vinculum firmware terminates each line of output with a single CR but most terminals require CRLF to move to the next line, so for proper display the response print code must add LF when it prints a CR. If done blindly then normal CRLF files will be double-spaced when read using the RD command, the 8052 prompt code monitors the previously sent character and only adds LF if one isn't printed. The HP-IPL/OS code does things a bit more crudely and simply doesn't add LF's if a RD command is issued.

The strategy for the 8052 image save code is to prompt for image size, convert the selected (hard-coded) number of blocks corresponding with the amount of data to save into two bytes of an LBA address to use to detect the end of the save, sets LBA to 0, deletes "DRIVE0.DSK" if it exists, opens "DRIVE0.DSK" for write, seeks to 0 [redundant], then for each 1KW block reads it from disk, swaps the bytes in each 16-bit word, and writes the buffer to the USB image file.

The strategy for the 8052 image restore code is to use the RD command to read the entire image file, capturing the bytes to a buffer. When the buffer fills with 2048 bytes it byte-swaps the buffer then writes the 1KW block to disk (automatically incrementing the LBA address which starts at 0) and loops to get more bytes. End of image-file is detected when a number of invalid reads of CR is detected (all VDRIVE2 responses end with CR). All HP-IPL/OS IDE disk writes occur in 1KW chunks thus IDE-compatible disk images are always multiples of 2048 bytes, so the prompt output represents an incomplete buffer and is not written to disk.

For core USB functionality from within HP-IPL/OS the interface implements four commands:

Reading and writing buffered stream files from HP-IPL/OS is perhaps the coolest of the new abilities (backup and restore is important but more drudgery than cool), the stream functions eliminate having to know how many bytes to read and/or write and both read and write files can be "open" at the same time. In fact neither file is actually open except when filling or emptying the read or write buffer so the user is free to issue Vinculum commands that internally open files even while streams are active. This prevents lost clusters on the USB drive should the drive be removed before issuing the write buffer close command, just an incomplete file (if remembered in time the drive can be reinserted to save the rest of the write buffer).

For stream functionality the interface implements six commands:

These functions are very generic to permit translating this functionality to other HP interface designs, for that matter it doesn't have to have anything to do with a minicomputer, could be just an embedded application that doesn't want to issue complex Vinculum commands to read and write files. By default when a file is "opened" for write it appends to an existing file, to overwrite simply delete the existing file first. Seek operations are not presently supported by these high level commands for a few reasons: papertape provides no such ability so the HP-IPL/OS streams don't need or use seek commands; when Vinculum commands are used to open a file for write the size must be increased or a lost cluster results (at least with version 3.66 of the VDAP firmware); if seek/read/write/close is really needed facilities are already in place to do it directly using Vinculum commands; and finally adding seek operations would complicate the stream algorithms for something I don't need.

Perhaps one of the hardest parts about using Vinculum commands is properly parsing the variety of error responses that might be issued, in this respect my 8052 code probably isn't great. For many errors (particularly those occuring while streaming) the code marks the error as unknown instead of figuring out precisely what the problem is, only the buffer open commands fully parse the error response to distinguish between invalid filename, file doesn't exist, attempting to open a directory, no disk, etc. It was a lot of code to write just so I could play with USB files on my mini - I wanted to get to the latter part and don't really care what the error was, only that an error occured. Another factor is try to parse every response byte  (even if ignoring) without invoking another time-wasting sync or clear operation, it's bad enough having to sync before doing buffer fills and flushes in case something else issued a command and didn't empty the response buffer. To avoid problems with uninitialized memory, each of the read and write filenames has an additional 3-byte check field which must contain certain characters or the name is considered invalid.

The following text descriptions describe the buffered streaming functions...

To open a buffered read file:
 Get the filename up to CR and put it in a read filename string
 (truncate past 12 characters but make sure there's a CR after the string).
 If an empty string was sent (just CR) then exit (abandons the read buffer).
 Call the sync code and select short/binary commands.
 Use the DIR command on the filename to validate the filename, this is a bit tricky
 but if a CR response is received before space then it's an error, size follows space.
If error returned by DIR then invalidate the read filename, set usb_error and exit.
Store numbers returned by DIR after space in read file size (4 bytes).
 If read file size = all 0 then it's a directory,
  invalidate the read filename, set the usb_error value and exit.
 Set the read file position to 0 (4 bytes).
 Call the buffer fill code, if an error occurs then exit.
 Mark the read filename as valid.

The buffer fill subroutine:
 Call the sync code and select short/binary commands.
 Open the read file, if an error occurs invalidate filename, set usb_error and return.
 Seek to read file position, if error close the file, invalidate, set usb_error and return.
 Read exactly enough bytes to fill the buffer, parse response to account for EOF response
   (don't set usb_error if "command failed" indicating EOF but detect other errors).
 Set buffer pointer to the beginning of the read buffer.
 Close the read file, if error occurs invalidate read filename, set usb_error and return.

To get a byte from a buffered read file:
 If the read filename is empty or not valid then
    send 0 to the HP, set usb_error and exit.
 If read file position = read file size (4 byte compare) then
    send 0 to the HP, invalidate read file, set usb_error and exit.
 Get byte from the read buffer at location read buffer pointer and send to the HP.
 Increment read file pointer, increment read buffer pointer.
 If read buffer pointer is past the end of the read buffer then call the buffer fill code.

To open a buffered write file:
 Get the filename up to CR and put it in a write filename string
 (truncate past 12 characters but make sure there's a CR after the string).
 If an empty string was sent (just CR) then exit (abandons the write buffer).
 Call the sync code and select short/binary commands.
 Use the DIR command to validate the write filename (see open read comments).
 If "command failed" returned that's OK (just means the file doesn't exist).
 If another error occurs or if an all-0 size returned (directory) then
    invalidate write filename, set usb_error and exit.
 Set write buffer pointer to the beginning of the write buffer.
 Set write buffer count to 0 (needed to know how many final bytes to write).
 Mark the write filename as valid.

To write a byte to a buffered write file:
 If the write filename is empty or not valid then set usb_error and exit.
 Store byte in write buffer at position write buffer pointer.
 Increment write buffer pointer, increment write buffer count.
 If write buffer pointer is past the end of the write buffer call the buffer flush code.

The buffer flush subroutine:
 If write buffer count = 0 then just return, nothing to do.
 Call the sync code and select short/binary commands.
 Open the write file, if error invalidate write file, set usb_error and return.
 Write buffer count bytes from buffer, if error then
    close write file, invalidate write filename, set usb_error and return.
 Set write buffer pointer to beginning of write buffer, set write buffer count to 0.
 Close the write file, if error then invalidate write filename, set usb_error.

To close a buffered write file:
 Set usb_error to 0.
 If write filename is empty or not valid then exit.
 Call the buffer flush code.
Invalidate the write filename.

The command to request the usb_error value just sends it back to the HP.

Whew! even expressed in high-level text it's a lot of code, and I mostly left out the parts for parsing the error responses except as they matter to the algorithm. The important thing to keep in mind if implementing a system like this, is no matter what the user does (including trying to read or write without opening and the memory is in a random state) the code must respond in a way that doesn't create disk errors, lock up the interface code or other undesired behavior. If an endless stream of zeros is returned causing HP-IPL/OS to lock up or cause a stack overflow that's OK, HP-IPL/OS already does that if an invalid stream is loaded and there are handy buttons on the front of HP minicomputer for halting and restarting if it loops forever thinking it's skipping an infinite leader, the computer is doing exactly what it was programmed to do. Sure timeouts could be added but it's easier to press halt, set P to 2 and press run then be more careful with what I type into the machine, but no harm done.

Project Notes

These notes are posted most-recent first, and subject to heavy editing. The notes represent what I think, not necessarily what actually is but I'm trying to hit it as close as I can. Theories will be modified as needed.

6/14/11 - sure enough, UFSIZE did cause issues when there are a lot of files, fixed as described below. Updated vdosext.ipl.txt and It's a little "risky" waiting forever (not really, at worse press halt, set P to 2, press run) but sometimes the VDRIVE just takes awhile to respond.

11/17/10 - I had a few bugs... my PIC code wasn't waiting long enough for VDRIVE responses, which fixed some of the issues I was having when there were many files in a directory. But VWRITE and VSAVE still failed, the problem was the VOWCHECK word was waiting for the first byte but assumed the rest of the response would follow without delay. Sometimes it doesn't, changed the code to use &WFVDR instead of  <VDR, that fixed it. There is similar code in the UFSIZE word used to get the size of a file, mainly used when copying it to an IDE/SFS disk (limited to 64Kbytes), but it does wait on the next byte after the first return issued by the DIR command (the firmware returns 100000 octal when busy) so leaving it be unless it causes trouble (if it does change the <VDR words in UFSIZE to &WFVDR to force it to wait for every byte). VDRIVE programming can be tricky! what works for one case might not always work.

Posted new vdos.ipl and vdosext.ipl files with the driver fix (below) and the VOWCHECK fix, updated with the new files - seems to work fine with the 8052 interface. Not extensively tested (booted, saved a system, reloaded, poked around a bit) but as the main change was the input driver, would stand to reason that if there was an incompatibility it wouldn't work at all... not that there aren't other undiscovered issues but given the nature of the bugs that were fixed, it's gotta be better. One thing I did notice... loading is faster using the old 8052 firmware and hardware than with the new (v0.32) PIC stuff... still got some optimizations to do, probably slowed things down more than I needed to (but better slow than buggy until I know exactly what I can get away with). The main advantages of the new PIC-based USB adapter is it has native papertape emulation and files can be attached using switches and an LCD without needing an HP-side dos, but for raw transfer speed the 8052-based design is quite nice... and has an IDE disk drive for running XDOS and SFS.

The new VDOS package behaves slightly differently, instead of an autostarting !VDOS word that patches itself to the IDE slot, the new package has a non-autostarting VDOS word that pops the stack and patches to the specified slot. Upon loading the package, the adapter slot is patched to the IDE slot if configured, otherwise sets itself to the PTR slot. This permits switching VDOS between the 8052 IDE adapter and the stand-alone adapter by simply doing (in my case) 23 VDOS or 12 VDOS.

11/15/10 - [snip no longer relevant info] I'm working on a new PIC-based USB adapter, it's basically operational except for tweaking. In the process I discovered a bug in the VDOS driver - requested data from the get vdrive byte, get next file byte and get error code commands are returned in the *same* command/flag cycle as the instruction, not a different cycle as implied by the VDOS driver code (that's why STC,C didn't work). The input driver should have been just a LIA port.


* receive one word from interface
JMP *-1

(probably) should be changed to this...

* receive one word from interface

Labels IF5 IF6 and IF8 are no longer used so they need to be removed from the patch table preceeding the code.

The extra STC causes the command line to glitch, which the 8052 code appears to be too slow to respond to but the PIC code sometimes picks up - although under some conditions (like when TBG is enabled) may also cause failure with the 8052 code too. Here's an experimental patch for existing builds until I can do more testing and clean up this little mess.

This probably explains the unrepeatable mysterious "glitch" I had during testing...

11/1/08 - I did further testing after reformatting my thumbdrive, wrote test code that performed 1024 unique file copies to the same file (512 each with both the "light" and "full" version of the interface code), and another test that wrote 1024 small files with slightly different content. No file errors at all. I did however discover that if the file was more than a few hundred files deep in a directory that the delay was enough to keep PVDR (print VDRIVE2 response) from printing anything when using VSHOW - modified VSHOW and VDIR so they wait for the VDRIVE2 to respond no matter how long it takes, and for VPROMPT (the RD command had a similar display "bug") issued 5 PVDR commands to give it 5 times as long to respond.

There was another glitch - just when I thought everything was perfect did a disk command and it locked up. Thinking I might have a power supply noise issue I added extra decoupling (47uF and .1uF capacitors in parallel) at both +5V/GND points going to the proto area of the Rev5 dev board - including at the point which the VDRIVE2 gets its power. Then later while doing a dump I bumped a cable and heard the drive reset - duh. Sometimes the most baffling bugs have simple explanations! But given the nature of a device such as the VDRIVE2 which absolutely depends on a valid state, I'm adding a note about decoupling to the connection notes.

10/28/08 - I think I'm going to take another approach to the VCOPY incident - not exactly pretend it never happened, but close. I've never had any trouble whatsoever with the image save and restore functions (at least after upgrading the firmware to v3.66), I've probably cycled it a dozen times accumulating about 100 megabytes of data writes. I've never had any trouble with streaming real files. I do think I'll get another thumbdrive and format it plain FAT with 8.3 filenames [which turns out to be impossible with normal tools - no matter what it lets the PC pollute it with LFN's] just in case that's got anything to do with it, verify files (especially after using VCOPY), back up my work, and if it happens again just rename the errant file so those sectors won't be used again. It's just too darn cool to let a little unexplained glitch scare me, I have to consider the rest of the operations which all work perfectly... I sure don't want to give those things up!

Doc file in the has been updated, it mentions the glitch (just in case it's an actual bug) but suggests a few simple steps to mitigate possible effects should the bug strike. Who knows, a cosmic ray might have picked that session to pass through and flip a bit... hmmm... put the VDRIVE2 in a metal case? For that matter the entire HP/IDE/USB controller should be encased (need to do that anyway).

10/26/08 - Posted the new vdosext.ipl "extra" package, contains wrappers named VREAD VWRITE and VCLOSE that warm-boot HP-IPL/OS if error (to keep further words on the same command line from executing), prompt before overwriting an existing file, and print messages for feedback. Also includes VLOAD for loading IPL files and loading and running ABS files (like XLOAD but for USB files), VSAVE for saving the current system to an ABS file on the USB file device, and UF2F and F2UF for transferring files as-is between the USB device and SFS files on the IDE disk. The UF2F word required a rather complex UFSIZE word that uses the DIR command and parses the output to determine the size of a file to know how many bytes to transfer and make sure the file isn't too big for the Simple File System (just shy of 64KB max). The package contains a new VCOPY word with more proper error-checking but ran into a problem - after a long debuging session it glitched while copying a small file, the size of the file was properly updated but the data itself was not written to the file. The next day the same test copy to the same file worked perfectly. Weird.

The zip file now includes a text file documenting this stuff, warts and all. I might discover things that change its contents so if the file date of the zip changes and the other dates didn't, I'm fiddling with the docs.

10/25/08 - The VCOPY example shows the basics but is too simple - there's no validation of filenames other than what the raw words do, and no protection from specifying the same file for both input and output which would destroy the file. Windows permits the same thing (i.e. type file > file) so I'm not too inclined to worry about "fixing" this in the HP/IDE/USB controller code, it's more than just a simple string compare since different strings can map to the same filename. The current interface code may need consolidated to eliminate having "full" and "light" versions (when I figure out what's important and what isn't), but as far as I can tell the current 8052 code works fine so it'll take more than it wrecking a file when given garbage input. Besides, the new "extra" VDOS words I'm working on include confirms etc to keep that from happening, coming soon along with a detailed doc file explaining what all this stuff does.

I'm not sure if anything should be done in the interface code to prevent opening both read and write streams on the same file, fixing that would take quite a bit of extra code (more than simply comparing, different strings can map to the same internal filename). Tenatively my inclination is to do nothing in the controller code to prevent (GIGO) and do the extra error-checking in additional words to add to VDOS. Windows does nothing to prevent entering a command like "type somefile >> somefile" (which churns away until the disk is full and the system goes down, oops maybe I shouldn't mention that :-) so if a major OS in use by billions doesn't check to see if the input and output streams point to the same file, I don't see why I should bother to do that in a system used by maybe a couple people on a good day... just don't do it.

10/20/08 - Here's a simple USB-aware IPL program for copying a file...

USBWRITE USBSTAT IFZ ;both files opened OK
DO #0 ;loop, change 0 on stack to <>0 to stop
MSBIN ;get byte from file
OVER IFNZ DROP ELSE MSBOUT ENDIF ;if not eof write to output
UNTIL ;end of file
USBCLOSE ;close output file

With this I can copy any ABS binary to "BOOT" and directly boot it using the IBL papertape bootrom and the "Auto PTR" hack. For my dual-driver systems it was a bit tricky, need to have the IDE boot code in memory even if not used so it won't prompt, and the precanned ABS save code ignores memory over 31KW. So instead (after redirecting to an ABS file using USBWRITE) I used PTZERO, AAOUT and ABSOUT to manually build the boot system binary the way I wanted it. Mainly though I wanted to see how long it takes for the stock papertape bootrom to boot a large ABS from USB - about 8 seconds, or about 7KB per second. Almost as fast as the raw SPI read speed (8KB per second).

10/19/08 - Put equates at the beginning of the interface source to make it easier to configure memory and port usage, the stream buffers are now set to 2.2KB each to be compatible with the original IDE interface design. I can tell hardly no difference from the previous 4KB buffers so I'll probably leave it that way. No significant code changes except now the USB errors are numbered more reasonably. A more significant development for my HP mini hobby pursuits is I can now boot my HP without the help of a PC. To do this I copy/pasted some HP hardware and USB subs into the previous "Auto PF2" program I was using to autostart (now called "Auto PTR") and wrote code to send a "BOOT" file to the HP if it's on the USB drive but rigged so subsequent resets run the disk interface unless I cycle the interface power, or flip my "auto" switch to Paulmon mode and reset which clears the mark it leaves to tell itself it's already run. Now my startup procedure is power the HP, power the interface with a thumbdrive in it containing the BOOT file, put 2300 in S, store, preset, IBL, preset, run (S bits 0-5 light up indicating successful load), then I press the reset switch on the disk interface, put 77600 in P, store, preset, run, it boots from the disk. Before I had to do the same things on the HP (except put 1200 in S) but also had to close the console terminal, move the serial cable to my PTR emulator, run HPSEND, close that, move the serial cable back to the console and run the console terminal. No more.

Here are some benchmarks with HP-IPL/OS 1.52 w/ VDOS 10/18/08 running on my HP2113 E-series mini and my 22mhz Paul Rev 5 board running the HP/IDE/USB interface code:

Save 2-volume 8.1 megabyte image file:    24 minutes 49 seconds (5689 bytes per second)
Load 2-volume 8.1 megabyte image file: 23 minutes 54 seconds (5907 bytes per second)
Save 46.7KB VDOS.ABS with 4KB buffers: 1 minute 2 seconds (772 bytes per second)
Load 43.1KB TREK.ABS with 4KB buffers: 39 seconds (1131 bytes per second)
Save 46.7KB VDOS.ABS with 2.2KB buffers: 1 minute 3 seconds (759 bytes per second)
Load 43.1KB TREK.ABS with 2.2KB buffers: 40 seconds (1103 bytes per second)
Save 46.7KB VDOS.ABS w/ 512 byte buffers: 1 minute 15 seconds (638 bytes per second)
Load 43.1KB TREK.ABS w/ 512 byte buffers: 51 seconds (865 bytes per second)
Perform 65536 bytes of raw SPI reads: 8 seconds (about 8192 bytes per second)

Some observations... only the raw SPI and image file times seem to represent the actual performance of the bit-banged SPI code, i.e. would be significantly affected by clock speed, better architecture etc. The stream operations don't get anywhere close to the native speed and are mostly limited by HP-IPL/OS itself (the SYSALL and ABSLOAD commands are written in interpreted code), and in the case of the 512 byte buffer tests, possibly by the speed of the VDAP firmware when opening and closing files (the smaller the buffers the more often it has to do that) plus the overhead of having to sync etc before every buffer fill or flush. A better test would be to time how long it takes to read/write streams using machine code, and how long it takes to boot the Trek game via IBL and the Auto PTR app - I bet it's much closer to the raw read speed.

10/18/08 - Everything seems to work including the new IDE status request command, but curiously when receiving USB data back from the interface I had to use a plain STC slot after sending the read request command, the usual STC slot,C request doesn't work. No idea why, STC slot,C works after giving the new status request command but doing the same for the read SPI function causes garbled data to come back despite using almost identical code procedures. I'm still not that good at HP I/O procedures so I count myself lucky to make it work at all, if it doesn't like C then so be it but probably need to find out what the deal is in case it matters. As far as I can tell it doesn't, just have to read the data back in a way that works.

To assist in possible porting efforts and to explore the effect of smaller buffers I made an alternate "light" version of the interface code, in the file. This version doesn't have the extra USB/SPI debug functions and confines ram usage to 3000-3FFF hex to avoid the port at 4000 in the original IDE interface design. The smaller 512 byte (vs 4 KB) buffers slow stream operations by about 20-30%, bigger buffers are better. Even with the debugging stuff removed the code is still far too big to load before the buffers in the current configuration, must load into flash rom at 8000 and above. The larger code version can also be configured for smaller buffers, edit the constants at the beginning of the stream code.

I don't know what to do with the stream error return values, right now they're powers of 2 except for the unknown error 255. Originally I wanted to allow for OR-ing the bits so multiple errors could be reported but presently nothing in the code does that. In general it's probably never a good idea to count on specific numbers whether from the stream functions or the IDE error value. Regarding the IDE error value, no idea what the returns are supposed to be (can't make the disk make an error) but I added code to set the value to 255 should an unrecognized command be sent to the interface so the new IDE status command will report something besides 0 when in an error-lock state.

10/14/08 - Streaming works! might need to tidy up a few things but essentially the new code does the main things I want it to - save and restore disk images off-line in debug mode, permit raw Vinculum commands to be given from HP-IPL/OS and parse responses, and read and write USB disk files using the normal HP-IPL/OS MS stream system. The present VDOS code was edited on the mini, saved to a USB file then copied it here. Only difference from the previous 10/12/08 version is USBCLOSE now reports an error number if one occurs, with the original "raw" USBCLOSE word renamed to &UCLOSE. Speed is good enough, dominated mostly by the speed of HP-IPL/OS itself. Can load an ABS file from USB in about a dozen seconds give or take depending on size, previously this took minutes using my pass-through PTR emulator to load the file from my PC, bigtime better.

I learned a couple things... if sending/receiving HP data in an added IDE command then it needs to exit with a jump to hp_disk rather than dx_done, otherwise it messes up the handshaking if another command comes along too quickly. I was able to load an IPL (slow) but the faster ABSLOAD failed, and doing USBSTAT too fast after another command returned 0 instead of the actual value. After exiting via hp_disk all was well. The other thing I learned is there's a huge difference between clr A and clr acc - AS-31 assembles the latter to a bit instruction and doesn't clear A at all. But still have to push acc to push A.

10/12/08 - I made a slight change to my IDE controller hardware - added a SPDT center-off switch and a 10K pullup to port F bit 2 connected to one of the switch terminals with the other connecting to the usual debug mode input and pullup on port F bit 0. Also added 470 ohm resistors in series with bits 0 and 2, mainly for convenient places to solder the wires from the switch but also to limit current should the bits ever get configured as outputs. Switch centered is normal operation, flipped towards the serial port is debug mode, and flipped the other way is ??? but thinking in this mode it should send the contents of a "BOOT" file to the HP as if the IDE interface were a papertape reader, then I won't have to swap serial cables etc to boot my machine.

The streaming code is coming along - after a bit of debugging I loaded MAZE from a file on the USB drive! normal computer users probably won't get the excitement, but for a minicomputer from the '70's it's fairly major (and would have been nearly impossible without the VDRIVE2 module with its built-in dos commands). I still have to write most of the write stream code but it should be simpler than the read code since there's no need to keep up with a file pointer for seeks and to determine EOF.

Here's the present "VDOS" word lineup...

; Core functions...
; !VDOS - autostarting word which patches itself to interface slot
; VSYNC - syncronize so responses match last command given
; VECS - syncs, selects extended commands and ascii numbers
; VCLEAR - removes last response if immediately available
; VPROMPT - runs a shell to enter commands and print responses
; VDIR - lists current directory of the USB drive
; "DIRNAME" VCD - changes to a new directory
; "FILE.EXT" VSHOW - lists file contents to the terminal
; "FILE.EXT" VDEL - deletes a file in the current dir
; byte >VDR - sends a byte to the VDRIVE2
; <VDR - pushes a byte from the VDRIVE2 (bit 15 set if not valid)
; "string" $>VDR - sends a string to the VDRIVE2
; "command" $>VCMD - sends a string to the VDRIVE2 plus CR
; PVDR - prints VDRIVE2 response if immediately available
; &WFVDR - wait for response and push 1st byte received
; Streaming functions...
; "FILE.EXT" USBAPPEND - redirects MS output to a USB file (appends)
; "FILE.EXT" USBWRITE - redirects MS output to a USB file (overwrites)
; "FILE.EXT" USBREAD - redirects a USB file to MS input
; USBCLOSE - closes a USB streaming output file
; USBSTAT - pushes USB streaming error status (0=ok)
; Example stream uses...
; Import an ABS file to disk file: "PROGNAME.ABS" USBREAD "PROGNAME" ABS2F

It looks like a bunch of stuff but presently only uses about 520 16-bit words of memory, or a bit over 1KB of code. In previous notes USBREAD was named MS<USB and USBAPPEND was named MS>USB but those word names are too easy to get backwards with data-losing results so named them so there'd be no mistake. USBWRITE simply calls VDEL then calls USBAPPEND to overwrite an existing file. Also, &WFVDR is the same as WF1B mentioned previously to wait for the VDRIVE to send back a byte, only machine-coded.

10/10/08 - The sync lockup was because of the code - what was triggering it was if a sync was called with no disk it completed, but if called again after a disk was inserted then it wouldn't complete until the power was cycled. The solution was to call the clear code in the sync loop when an incorrect response received, updated source and the algorithm description. Upon testing on my HP2113E I found it still wasn't working correctly, however the source of the problem had nothing to do with code and probably has been "messing with me" all along - when the HP is powered and the HP-IDE interface is off, leakage through the level shifters raises the supply voltage to about 1.4V. I already had a 330 ohm resistor from +5V to ground to drain it to this point, which is low enough to permit the dev board to reset, but not low enough for the VDRIVE2. So added another 270 ohm resistor from +5V to ground to drop the leakage to about 0.6V, problem solved.

10/9/08 - [edit] need another IDE interface command which returns a error value for USB operations...

137xxx - return USB file error value (0=no error)

Error values TBD but will probably mirror the errors returned by the VDAP USB firmware plus other conditions such as streaming when a buffer hasn't been defined. The important thing is to make sure nothing goes wrong no matter what garbage is specified - if a write file is invalid streaming to it anyway should simply discard the data and set the appropriate error bit, invalid reads should return 0, etc, with results no worse than what already happens if a MS word is used on an invalid stream and in some respects better - getting a stack overflow and a prompt from an invalid LOAD is messy but much better than what HP-IPL/OS normally does in such a situation (lock up, requiring halt and run from 2).

10/8/08 - Improvements to the debug-mode functions... the dos prompt now automatically adds LF after CR if not present to work properly with regular VT100-type terminal emulations, the save option now offers a choice to save up to nine 64-file SFS volumes for use with a pure-IDE setup, and the image save function now deletes an existing image file first to avoid lost clusters. I think the general solution to that problem is don't open a file for write unless intending to actually write data past the current end of the file, shouldn't be a problem for the streaming functions but this phenomena would complicate the design of a disk emulator (none immediately planned but has been discussed) where file growth does not occur and obviously the existing disk image file can't be deleted. Or simply ignore the lost cluster that'll always be present when using such a system (perhaps a firmware update will fix this). The controller source has been rearranged in preparation for adding the streaming code, added IDE command word hooks but presently they're no-ops pointing to comments (subject to change). Got an autostarting-version loaded so tonight I can test it to make sure it's still actually a disk controller, need to fix VPROMPT to make sure ascii command mode is enabled and fix the automatic addition of linefeeds. Trying to get all the basics solid before starting on the streaming file code to avoid distraction from having to fix goofy stuff.

10/7/08 - Played with the new controller for a couple of hours... everything seems to be working fine now (the noise problem seems to be fixed) except for the intended lockups when I mess up and send the VDRIVE2 into a blinking-led fit, not getting the commands right has that effect. After several tries and many lockups and trips to the PC to fix the resulting file system errors I finally managed to write a program that writes a line of text to a file. Had to add a word to wait for a byte no matter how long it takes but it probably won't be part of the "official" VDOS package as I don't want to encourage programming in the Vinculum command set from HP-IPL/OS, way too tramatic! Here's what it took to append "HELLO FILE[crlf]" to a file named "TEST.FILE":


...needless to say this simply won't do :-) (but nice to be able to) The main reason for this test was to see how fast it takes to open a file, write data to it then close the file - the first time it takes a few seconds but after that there's no significant delay, which is very good since to implement streaming it'll have to repeatedly open and close the output file to write the buffer when it fills up. All this stuff needs to be in the controller code so the user won't (or shouldn't) have to worry about making a mess of the file system like I did while testing. With streams the above code reduces to simply "TEST.FIL" MS>USB "HELLO FILE" MS$OUT MSCRLF USBCLOSE and even if I forgot the close the buffer simply wouldn't get written rather than corrupting file system. Repeated appends to an already open file would be simply "string" MS$OUT MSCRLF (if a text line, adding binary values would be just [number] MSBOUT). Lots of little details need to be worked out - like if a new output file is buffered while another one is active it should do an implied flush of the existing buffer to the original file first, and if a file isn't buffered, reads should return 0 and writes should be no-ops.

I noticed a minor problem - when using the debug-mode save option to overwrite an existing image file the filesize is correct but fsck.msdos complains that the cluster chain is slightly longer so truncates it to match the filesize. No data is lost and if the file grows the extra cluster is used and no file system errors are reported. This appears to be how the current VDAP firmware works, opening a file for write always allocates an additional cluster, simply entering OPW TEST2.FIL followed by CLF TEST2.FIL without writing anything to the file results in the following when the file system is checked...

dosfsck 2.11, 12 Mar 2005, FAT32, LFN
File size is 0 bytes, cluster chain length is > 0 bytes.
Truncating file to 0 bytes.
Free cluster summary wrong (55417 vs. really 55418)

Confirmed to happen even with a freshly formatted thumbdrive. This doesn't seem to be a data integrity issue, just a wasted cluster I'd never know about except for looking. Another thing to watch out for - the VDAP firmware knows nothing about long file names, if a VDAP REN command is used to rename say an image file, the PC may continue to use the old name and the DRIVE0.DSK file copied over might not be the one just created. A way around this is to only have one version and keep other versions on the PC rather than on the thumbdrive. The rule seem to be don't use the VDAP REN command to rename files copied from or renamed by a PC, only use it to rename files created by the VDRIVE2 itself. Also helps to have only one DRIVE0.DSK file varient on the thumbdrive, otherwise the DRIVE0.DSK file copied from the thumbdrive to the PC might not be the one just saved. This effect was discovered when I copied what I thought was a newly created image to my PC but when I booted it under sim found it to be a previous version I had renamed. Not a big deal just something to be aware of.

Anyway... the present to-do list seems to be...

10/6/08 - It's more than a theory now... I implemented the first part of the specs below (no streams just essential I/O) and it works, from my minicomputer I can log onto the thumbdrive and list directories, change directories, list files, etc. It's almost solid as a rock - but if the VDRIVE2 stops accepting data for any reason the drive interface locks up, like it did once after doing a lot of disk I/O. Should probably put a time-out on this, and noticed the SPI output lines were being switched to input by the disk code, which could cause the SPI lines to pick up noise from the high-current level shifters, changed but haven't retested yet. Also changed but haven't tested the new status request command, now code 105xxx. Latest controller code and VDOS words posted, which in addition to the VSYNC to VPROMPT words also has a PVDR word that prints the current response from the VDRIVE2.

One goofy little glitch I didn't think of - the VDRIVE2 sends CR line ends so I added a LF when one comes along, only that causes files listed using the RD command to be double-spaced so added a variable to control whether or not that happens. When RD is used it turns off the added LF. The present implementation isn't perfect and results in some garbage around the RD command from missing linefeeds, a better way would be to add the LF only if one does not appear after CR.

I'm anxious to add the streaming file I/O commands! Having that would have saved me the 15 minutes it took to extract the entire volume and boot in under sim to extract the VDOS code I wrote while using the mini, when I could have just typed "VDOS.IPL" MS>USB MASAVE USBCLOSE and had the file in seconds.

10/5/08 - Working on adding these new IDE commands...

120xxx - read one byte from the SPI interface, if not valid set bit 15 (and clear lsb?)
121bbb - write one byte to the SPI interface, wait for byte to be accepted
122xxx - call VDRIVE2 sync code
123xxx - call response clear code

...and HP-IPL/OS words like these that use the new commands...

VSYNC - sync with the VDRIVE2 (*)
VCLEAR - clear response from the VDRIVE2 (*)
n >VDR - send one byte to the VDRIVE2 (*)
<VDR - push one byte from the VDRIVE2 (*)
"string" $>VDR - send string to the VDRIVE2 without CR
"string" $>VCMD - send string to the VDRIVE2 with CR
VPROMPT - a VDRIVE2 prompt

The words marked (*) are specific to the IDE interface, so the rest of the code can be written more generically and reused if the interface is implemented differently. This is enough to get a dos prompt and even do simple file processing, but a more structured approach is needed to safely handle MS streams...

130xxx - open buffered input file - must follow by sending filename[cr]
131xxx - read one byte from buffered input file (if EOF or not open returns 0)
134xxx - open buffered output file - must follow by sending filename[cr]
135bbb - write byte bbb to buffered output file (if not open then a no-op)
136xxx - close buffered output file (if not open then a no-op)

...along with MS-related words (these would be interface-specific)...

"filename" MS>USB - redirect MS output to buffered USB file
USBCLOSE - close USB output file (stops buffering, MS out left redirected)
"filename" MS<USB - redirect MS input to come from buffered USB file

That's more like it, as far as HP-IPL/OS is concerned it'd be like papertape - to load an IPL file I'd do "filename" MS<USB LOAD, to punch the system to ABS I'd do "filename" MS>USB SYSALL USBCLOSE, to load and run an ABS file I'd do "filename" MS<USB RUNABS and all the other normal MS stream stuff, including things like "filename" MS<USB "SFSFILE" ABS2F to import an ABS file to a SFS binary. This sounds reasonable, although some of the words are specific to the IDE interface the general functions are not and VDRIVE2-aware app code could be reused regardless of the hardware/firmware implementation.

Previous similar (but less defined) speculation removed except for the thought that maybe I should add a switch to PF bit 2 so when the switch is activated the interface sends the contents of a boot file in a form the HP mini's papertape bootrom can load, then I wouldn't have to swap cables and send the HP a kickstart program from my PC to make it go.

10/4/08 - first reasonably stable version of the new controller code with core image save and restore fuctions accessible from the controller's debug menu. The save option gives the choice of saving the boot system and a single 64-file SFS volume (4.1MB) or an amount of data equivalent to a 7906 disk platter (9.6MB). These choices are convenient for me as I typically only use one volume, and I format my IDE disk to 7906 specs so I can swap disk images between the IDE disk and the SimH HP2100 simulator. The restore option restores however many complete 1KW blocks are in the image file, so it doesn't take as long to load smaller images. A SPI/USB debug menu permits sending and receiving individual bytes to and from the VDRIVE2, and a console "dos prompt" for directly typing commands and receiving output. Also has an option which calls user-specified subroutine addresses - while useful for debugging, need to make it go away unless manually enabled in the source. Wrote the initial content of this page.

10/3/08 - finally got it to save an image file properly after upgrading the VDAP firmware. The original 2.08 firmware appeared to replace some of the output to the output image file with garbage and occasionally would lay down with the led blinking away. Some of the effects were probably from my bugs but after upgrading I was able to save a bootable image using the same code that previously didn't work.

9/28/08 - got a VDRIVE2 USB disk interface module and began coding the new controller code.

Last mod Nov 1 2008