Technical notes for the 7906 sim and HP-IPL/OS
==============================================

Some of this material is repeated elsewhere and this doesn't replace
other docs, rather fills in a few gaps and summarizes the file system
and present software implementation.


Importing and exporting editor-format files
-------------------------------------------

The IPL files, IPL apps (beginning with $), and the # NOTES file on the
disk are in the editor's "array" format, where each line of the file is 78
characters long plus a CRLF, with zero bytes following the text data. File
sizes are a multiple of 3460 octal bytes (the size of each 23 line page).
There should be no need to export the IPL files or apps unless changed
(regular text versions are available elsewhere), if exporting is needed use
the SFS/EDIT build then halt (control-E) and attach PTP to an output file,
continue and "FILE.IPL" LDFILE MASAVE then halt and detach PTP. Same for
# NOTES and the $ files. If the export text option is used from the
import/export submenu, the space padding and trailing zeroes will be
included in the output file. The # SUMMARY file and sources for the BCS
games are in regular text format and can be extracted using the export
text option without effect. Export text exports the file exactly as it
appears in the disk file.

To import a text file in a form that can be edited run the SFS/EDIT build,
halt and attach the file to PTR, continue and enter MALOAD, if the file does
not have ~TERMINATE~ by itself on the last line it will hang after loading,
in which case halt then enter run 2 to start (the general recovery procedure
whenever something hangs). Once back to the ? prompt enter "FILENAME" SVFILE
(or "# FILENAME" to display on the menu) to save the editor-format file.
Note that the editor is limited to 782 lines or less. To import regular
text files (less than 64Kbytes) for reading only (not editing) use the
menu's import/export submenu.


Additional menu usage notes
---------------------------

If the Load ABS option key 3 is inadvertently pressed, press any key then Y
at the halt prompt but instead of attaching a file enter run 2 to rerun the
system. At some point I might modify the menu to return to the menu unless
Y or N is pressed at the first prompt.


Creating a new empty disk sim
-----------------------------

Basically, to create a new blank simulation except for the boot system,
all that's needed is a copy of block 0 and the HP-IPL/OS boot system.
To create an ABS copy of the boot system, go to a prompt, control-E to
halt and at the sim prompt enter attach ptp hposmenu.abs then enter c to
continue. Enter again if you want to see the ? prompt but not necessary.
Enter SYSALL to write the build to PTP. Now to save block 0. At the ? prompt
enter the following (exactly... we're doing low-level disk stuff here):

OCTAL DEFINE TEMP
0 0 SBLA @BLK GET R-1K
@BLK GET DUP 1777 ADD +DO
INDEX GET MSWOUT +LOOP END

Control-E and enter attach ptp block0.bin then enter c to continue and
enter TEMP to run the word entered in from the above code. Control-E and exit.
The block0.bin file should be 2Kbytes in length. Move the hposmenu.abs,
block0.bin, and boot_7906.sim files to a new directory to build the new sim
in, along with any ABS or other files you want to import into the new sim.

If desired run the stock sim again and use the ABS and text export functions
to export any binaries, IPL apps/files or text files you want to move to the
new sim. Note.. as described in import/export notes, files in editor format
will contain zero bytes when exported so might not open in a PC editor, but
this will preserve the file in editable format without having to do all the
MASAVE/MALOAD stuff.

In the new dir, "run" boot_7906.sim in SimH HP2100. Nothing useful will happen
but creating an empty disk image and setting up the slots. Control-E then at
the sim prompt enter load hposmenu.abs then enter run 2 to start it, an empty
menu should pop up, press 1 for the prompt. Enter DGEN to write the system
to disk as the boot system (but won't be bootable yet).

Now to "format" the disk...

Enter XINIT then enter 3 for the number of volumes, then enter 26 for the
(octal) number of files in the last volume.
Enter "NAME" MKVOL to create the first volume, change NAME to what you want
displayed in XDIR listings, 16 characters maximum.

Now to make the disk bootable by copying block0.bin to block 0...
Enter the following code at the prompt:

OCTAL DEFINE TEMP
@BLK GET DUP 1777 ADD +DO
INDEX MSWIN PUT +LOOP
0 0 SBLA @BLK GET W-1K END

Control-E then enter attach ptr block0.bin then enter c and enter TEMP to
read block0.bin and write it to block 0 of the disk. Control-E and exit.

"Run" boot_7906.sim again, this time it should boot. The import/export
submenu (option 6) can be used to copy in whatever you want, to display
on the menu ABS binaries should be named @ FILENAME and text files should
be named # FILENAME to display on the menu. If importing IPL apps to run
from the menu they must be named $ NAME where NAME is the exact name of
the first word, such as $ FED, $ MAZE, etc. Otherwise they won't work.
Leave out the @ # $ prefixes to not display the files on the menu, these
can be loaded using "FILENAME" XLOAD or viewed using "FILENAME" XSHOW.


Customizing the menu build (and HP-IPL/OS builds in general)
------------------------------------------------------------

If interested in customizing the menu the source is in DM2.IPL, feel free to
edit as needed (if you want to play with IPL code which is stack-based and
unlike probably any other programming language except a passing resemblence
to FORTH). Before messing with it might be a good idea to save the original
boot system to a file - at the prompt enter "STOCKMENU" XSAVE - and save
edited versions to another filename such as DM2A.IPL or similar. Once a new
menu source has been prepared, go to the prompt and enter "!DMI" 0 STASH to
save the custom !DMI word (which avoids running the menu in case the build
is saved to PTP using SYSALL and booted without a disk present - if not a
factor then don't bother with STASH and FETCH and forgetting the stock
!DM word), enter FORGET FED and confirm to remove the existing menu,
"DM2A.IPL" XLOAD to load the new version of the menu, enter FORGET !DM and
confirm to remove the stock !DM autostart word then enter 0 FETCH to restore
the original !DMI word. While testing mods, save the build to a file, i.e.
"MODMENU" XSAVE (do "MODMENU" XDEL first if it already exists), the modified
version can be tested by doing "MODMENU" XLOAD from the prompt. When all works
as desired then DGEN can be used to write the modified system to disk as the
default boot system. That in a nutshell is how HP-IPL/OS builds are updated...
but further notes on the process, if loading additional IPL's into the build
always do WORDS afterwards to keep tabs on free memory - there's no checking
for overflowing the dictionary - and the STASH/GET trick for preserving words
after the FORGET point only works on high-level words (EXPLAIN WORDNAME must
show code), and only if the specified block (0 in the example) is available.
The default boot system has 3 blocks numbered 0-2, whereas the SFS/EDIT build
has only blocks 0 and 1.

The other way to update the system is to use the sources and scripts in the
provided mkdisk.zip file, to use the mkdisk directory must be moved into the
hpiplos directory of the 1.51(x) HP-IPL/OS main archive, alongside the abs,
ipl, and scripts directories which are referenced by the build scripts.
Updated versions can be tested using the stock menu's Load ABS function or
break to the sim prompt, load filename.abs and run 2. This method is easier
if you wish to use a PC editor to work on the code (make sure sources are in
CRLF format if editing under Linux) and semi-automatically handles all the
!DISK/!DMI stuff but as written requires that the menu build be built from
scratch starting with the kernel. Once used to HP-IPL/OS procedures (and if
you can stand the crude editor) it's easier to make minor changes on-line,
but the build scripts provide an opportunity to edit almost anything and
include/exclude options as needed. Watch out for page errors, in HP-IPL/OS
machine code has to be arranged so it does not cross a 1KW page boundary.


About the Simple File System (SFS)
----------------------------------

SFS was invented by Bob Shannon (who also invented the original HP-IPL/OS)
to provide a simple way to store and retrieve files on a HP-IPL/OS disk.
But first some background about how HP-IPL/OS disk support works...

Presently disk drivers exist for 7900, 7906 and IDE disks, and with a bit
of hacking (as with the 7906 sim) it is possible to support multiple disk
types and autodetect the one to use by looking in high memory to see what
disk the build was booted from (if any). The 7906 sim contains 7906 and IDE
disk drivers, on startup the !DISK word checks to see what it was booted
from, prompts if it can't tell, and sets a %DT variable accordingly.
To avoid running the disk menu when no disk is present the disk menu's
stock autostarting !DM word has been replaced with !DMI which only runs
the menu if a disk was detected or selected. This "combo" method isn't
exactly "proper" procedure (normally only one disk driver is loaded)
but permits me to boot the exact 7906 sim disk image on my HP2113E IDE
disk system by copying 7906.dsk to an IDE disk using a USB adapter and
a dd command (conv=swab).

No matter what disk driver is used, data is stored on disk in 1KW chunks and
always in the same logical location, disk space permitting. Thus SFS, XDOS
and other disk software generally don't care what kind of disk it is, and
a new disk can be supported by writing a driver for it. There are only
a few functions the driver has to perform...

  Set the logical 1KW block address (high low SBLA calls this function)
  Load a 31KW binary into main memory from the set block (LDFB)
  Load 1KW from the set block into memory (address R-1K)
  Save 1KW from memory to the set block on disk (address W-1K)
  Check the drive status (<IDE pushes status to the stack)

Each R-1K and W-1K operation increments the logical block address to avoid
having to reset it before each read or write of larger files.

The status check is optional, often (as in the case of the sim) the <IDE
word is modified so it simply pushes 0 and doesn't actually check status.
This is because my older HP interface board isn't compatible with the status
method used by the present IDE interface hardware/software. Generally if an
error actually occurs there will be other effects (like it not working).

In addition a DRV variable is defined in zero page (along with the other
drive variables) which the driver uses to tell which drive to access, the
fixed platter of 7900 and 7906 drives are indicated by adding an offset
(4 or 10 octal) to the drive number. Low-level HP-IPL/OS disk access is
always based on a 1KW block size. The IDE driver and interface hardware
supports this natively but real drives typically require multiple accesses
for each block, taken care of in the driver so HP-IPL/OS sees the disk as
an array of 1KW blocks regardless of the actual disk geometry. For more
information and actual implementation details refer to disk.ipl, ide.ipl,
7900.ipl, 7906.ipl and hpos_disk.txt from the main archive, online at:
http://www.infionline.net/~wtnewton/oldcomp/hp2100/files.htm

Back to SFS. The Simple File System provides for a boot extension in the
first 6KW (blocks 0-5) if needed to load the boot system using a disk bootrom
(IDE disks don't use a boot extension, the IDE bootrom boots a SFS boot system
directly), block 6 contains the master volume index which provides entries for
up to 64 "volumes" (directories) and specifies where each volume's directory
block and file space is located on disk, generally block 7 contains the first
directory block, and blocks 10-46 (octal, all addresses specified in octal)
contain the boot system loaded when the disk is booted. Each directory block
contains entries for up to 64 files, each entry specifies the filename, size
in bytes, load and run addresses for binary files, last access point, other
attribute bits (which aren't presently used) and most importantly, where the
file begins on disk - not an actual number but implied by the position.
The first file will be in blocks 47-106, the next file in blocks 107-146 etc,
with the last file of a 64 file volume in blocks 4007-4046. The directory
block for the next volume is in block 4047, with blocks 4050-10047 for the
next 64 files, repeating until disk space is exhausted or all 64 volumes
are specified.

This is the recommended arrangement but except for the boot system, where
things are located on disk is wholly dependent on the numbers in the volume
index in block 6, and each directory block. A major advantage of sticking with
the standard format is data can be easily recovered by existing utilities
should the beginning of the disk were to be overwritten by a coding accident.
Or if I wanted to, I could use XINIT to "reformat" the 7906 disk image to
IDE specs then run VRECOVER to recover existing volume directories.

The IDE driver uses the DRV variable (drive number) to offset the actual block
address to store as many 4096 file virtual disks as will fit on the disk. Due
to math limits a maximum of about 2 billion 1KW blocks can be accessed, enough
for thousands of virtual drives but practically limited by the maximum sized
IDE disk which can be accessed by the old "LBA" method used by the 8052-based
code supplied by Paul Stoffregen, see: http://www.pjrc.com/tech/8051/ide/
I'm using a 40 gig IBM Deskstar drive which gives me up to 165 virtual drives
if I wanted. But practically totally overkill, I've never bothered to reformat
past the 10 megabytes or so provided by a single 7906 platter, enough for 150
files. I rarely have over 64 files even under sim and so far have yet to
create another 64-file volume on real hardware, at the moment I'm using
about 2.5 megs of my 40 gig drive.

In large disk environments, wasted space from the simplistic SFS allocation
scheme is not an issue. In the case of IDE not even close to an issue. For
smaller disks however SFS does have limitations, a 7900 drive only has room
for 36 files per platter. Still quite useful for storing 31KW binary images
(the most common use of files anyway) but not very useful for storing a bunch
of small files. One way to overcome this at least to a degree is create a
mini file system within each 32KW file slot, one way is using the lib2.ipl
"library" system option which permits many words, IPL files, and IPL apps
to be stored in a single tape-like file (only the last file can be modified).

The apparent visible origins of SFS can be traced back to a series of postings
to the ClassicCmp "cctalk" mailing list in July 2003, in a thread entitled
"Simplest (practical) file system?". SFS truly lives up to its name, and for
what I use it for it certainly is practical, and more than anything made it
possible to write a HP 21MX DOS for HP-IPL/OS without having to resort to
swapping in overlays and other complex mechanisms.


About XDOS and the SFS file-handling words
------------------------------------------

XDOS is a HP-IPL/OS dos that implements most aspects of the physical SFS
disk layout. The system as presently implemented is quite useful and can be
used to store just about any slot-compatible 31KW binary rather than having
to load it from an ABS file connected to PTP. The main limitations of running
binaries from disk is they overwrite the entire 31KW memory space, and the
load mechanism presently always runs from location 2 (for binaries both the
load and run addresses are 2 unless SW is written to do anything else).
It's quite easy to put something in locations 2-3 to jump to the true start
address so that's not really a problem. When HP-IPL/OS with XDOS runs a binary
it copies itself to alternate memory and puts a small "swapper" program in
high memory at 77000, when this code is run it swaps main and alt memory
and runs from 2, rerunning HP-IPL/OS and leaving whatever was running in
alternate memory where it can be saved. This, along with a bit of patching,
permits HP BASIC and MSU BASIC programs to be saved to disk by simply saving
alternate memory to a disk file. When running unmodified HP binaries the
system has to be halted and 77000 run manually, but that's easy enough and
is much preferable to having to reboot. In addition to loading binaries,
the XLOAD command can also redirect files into the console, executing the
commands as if they were typed (like LOAD does with files attached to PTR).
An XSAVE command writes the HP-IPL/OS system in memory to a binary disk file.
XDOS provides many commands for initializing, navigating and maintaining the
file system, transfering alternate memory to and from files, and copying
files to and from PTP/PTR. ABS files can be imported directly, importing
arbitrary files requires specifying the exact number of bytes to transfer
(it's a stream that may include zeroes so it has to be told how much, the
7906 sim's menu contains extra import code that assumes text and stops when
the trailing zeroes are encountered to avoid specifying the size), and
exporting a binary in ABS format requires using extra code (built into
the 7906 sim's boot system).

Note that XDOS' operations tend to be all-or-nothing. There is no way to
load say an overlay, directly read or write a data file (other than storing
data in alternate memory to write to and retrieve from a file in one chunk),
or redirect the HP-IPL/OS "MS" I/O streams (mass storage, normally PTR/PTP)
from or to a file. For many things like the disk menu or if an app doesn't
use data files, "advanced" file access is not required. But for things that
need it there are the SFS file-handling words in sfs.ipl (on the 7906 sim
broken into SFS1.IPL and SFS2.IPL to permit compatibility with the editor).
Often these words are refered to as simply SFS, as XDOS isn't quite SFS and
only implements disk layout and file system utilities. SFS[x].IPL implements
the file-handling aspects of the Simple File System and provides buffered
access to up to four open files, each with individual access pointers.
SFS permits reopening files with the previous file pointer intact so an
application can continue a file operation without having to keep the file
open, or even between reboots. SFS makes it fairly trivial to overlay
binaries in alternate memory by copying an ABS file intact to a disk file
then redirecting the file into HP-IPL/OS' existing ABSLOAD word. This
technique can be used to run antique HP software that use overlays such
as pass 2 of the FORTRAN compiler. A simpler application of SFS is the
editor's LDTXT utility, which can be used to load a plain text file from
disk and convert it to the editor's array format by redirecting it into
the MALOAD utility. Personally I don't use SFS that much, I tend to have
simple needs addressed by XDOS (why I wrote XDOS) so often I leave SFS
out of the dictionary to save memory, but when I need the extra abilities
(sometimes it's vital to do a particular thing) it's as simple as loading
a build which contains the SFS words. One size does not fit all, but that's
why each HP-IPL/OS build should be taylored to the job it needs to perform.

It's possible to write a unified dos instead of the present separate
packages, but in thought experiments it seems to always either be just as
big as the present XDOS/SFS mix, take away memory for simpler apps, or don't
provide all the utilities I need to do what I do and I'd still need XDOS.
Another factor is XDOS uses only one 32KW alt memory page thus can run on
a 64KW machine, whereas SFS requires up to 192KW of memory. A disadvantage
of mixing the two systems is XDOS commands that write to disk are disabled
if a modified SFS file is being buffered, I can live with that. Most SFS apps
close the files they use when done, except for the library and it's easy
enough to close it to write changes to disk, which should be done anyway
before altering the disk using non-SFS methods. There are no SFS functions
specified for many tasks (saving binaries, creating/renaming/deleting
volumes, renaming files, etc) so even a unified approach would still need
the "don't mod" flag to make sure buffered disk contents are not changed.

Operating system design, in particular for a hobby system, is a personal
thing, and in the case of HP-IPL/OS, has been a slowly evolving mix of what
seems to work, but isn't exactly as elegant as it could be if one were to
set out and re-implement it all from scratch. The same can be said about
most operating systems, which hardly ever please everyone. Nor should they,
different tasks require different methods, to make an OS totally useful for
every application would bloat it to the point of no longer being useful for
other things, a situation now faced by modern OS's that try to do everything
and in the process become too bloated and slow for simple things that used
to be easy. When I have to write a simple app, often HP-IPL/OS reading from
PTR and writing to PTP gets the job done faster than trying to make a "real"
programming language work. For numeric stuff MSU BASIC works well.

--------------------
Last mod 4/18/08