Version 4.00
10 July 1992
David Quarrie
Brian Troemel
CDF Computing Group
Fermilab MS318
Batavia IL 60510
Tel: (312) 840-3975
DECnet Address: B0HOST::QUARRIE
B0HOST::TROEMEL
BITNET Address: "QUARRIE@FNALB0"
"TROEMEL@FNALB0"
1 PREFACE
YBOS is based on two previous memory management systems,
BOS and ZBOOK, developed at DESY and CERN, respectively.
Version 1.00 was designed and implemented by:-
V. Blobel
II. Institute fuer Experimentalphysik
University of Hamburg
Notkestrasse 85
2000 HAMBURG 52
WEST GERMANY
This Reference Manual describes the CDF implementation of
YBOS which has been modified to run on a VAX or IBM and has
several extensions added.
2.1 Differences between Version 4.00 and 3.00
(a) A Bank Subset functionality has been added. Subsets
of banks are defined by lists of bank numbers which
are associated with a single bank name. One Bank
Subset is allowed per bank name. The members of a
subset for a particular name are stored in a work
bank maintained internally by YBOS. Bank subsets are
used in conjunction with BSWRIT and BSREAD. BSWRIT
acts just like BLWRIT except that only banks which
are subset members for the current name are written
to the logical record. BSREAD acts just like BLREAD
except that read banks are placed in the appropriate
bank subsets. The following subprograms (described
latter in this document) have been added: BSBADD-add
list, BSBALD-add dynamic all, BSBALS-add static all,
BSBDRP-drop list, BSBPRG-drop all, BSWRIT-write
logical record,BSREAD- read a logical record.
(b) The internals of YBOS have been extensively reworked
to improve performance. Thus most YBOS operations
(such as BLOCAT or BMAKE) are 50-100% faster, whilst
some others (such as sequences of BMAKE and BLOCAT
with many banks of the same name) are significantly
faster (up to 1000%).
(c) The documentation has been updated to include all
user-accessible YBOS Subprograms.
2.2 Differences between Version 3.00 and 2.00
(a) This document has been renamed to be a Programmers
Reference Manual to better reflect its scope and
intended audience. It has undergone a major
revision, reflecting a change in philosophy whereby
the details of the YBOS Bank formats (both for named
and work banks) are better hidden from the casual
user. Additional subprograms have been incorporated
so that the user data within a bank, or components of
the bank header may be accessed in a transparent
fashion.
(b) A Bank Type has been made an integral part of a YBOS
Bank. This describes the internal format of the
Bank, whether it contains Integer or Real*4 datawords
etc., or a mixture of datatypes. It's purpose is to
allow the automatic conversion of datatypes on
machines of different architecture and internal
representation of datatypes. The implementation is
backwards compatible in that the basic Bank Structure
is unchanged and all subprograms from previous
Versions are unchanged.
(c) Extra subprograms have been added to YBOS to deal
with the Bank Type and to aid access to the data
within a Bank or components of the Bank Header (Bank
Name etc.).
(d) Some User-accessible locations have been allocated
within YBOS. These locations may, for example, be
used to save work bank indices and to allow different
modules within a program to access the same work
banks.
(e) Long Alias Names may be declared for Named Banks. A
Bank having both an Alias Name and a Basic Name may
be located by either.
(f) Various Modes of Operation for YBOS may be specified.
These Modes allow, for example, for Work Banks to be
locked in place temporarily, or for the automatic
reporting of error messages to be disabled.
(g) All YBOS subprograms may be accessed as Subroutines
or Integer Functions, where the function value is
either a Success/Error Flag or a Bank Index. In both
cases the Success/Error Flag is also located in the
2nd element of the appropriate YBOS array, and is
therefore backwards compatible with previous Versions
of YBOS. Subprogram BERROR has been added to allow
the Success/Error Flag for the last YBOS call to be
examined as an alternative either of the above.
(h) Error Codes and Function Values are specified as
Integer Parameters rather than numbers. These
Parameters are defined in a Section of this Manual
and an "Include" File for insertion into each program
module accessing YBOS.
(i) In an attempt to rationalise names, the convention
has been made that YBOS subprogram names will
commence with the letter B unless they refer to work
banks, in which case they commence with W.
Exceptions to this are NBANK, MBANK etc., which are
retained for backwards compatibility. However the
Disc-based routines have been renamed to commence
with a B rather than a Y.
(j) MLINK and NLINK have been modified such that they
return the Index to the lowest numbered bank of the
specified name if presented with a negative bank
number.
(k) BCOPY and BRENAM have been added. BCOPY copies the
contents of a named bank to another bank, and BRENAM
renames a named bank.
(l) BKEEP has been added to enable all banks to be
dropped except for specified banks.
(m) BAREAD and BAWRIT perform similar functions to BLREAD
and BLWRIT except that the destination (or source)
for the logical record is an array rather than being
an I/O channel.
(n) BEREAD and BEWRIT perform similar functions to BLREAD
and BLWRIT except that the Logical Record is read
into (or written from) an Array rather than a YBOS
array.
(o) YBERMS and BWARN have been added to enable error
messages to be reported.
(p) Several new subprograms for Bank Printing have been
added (BAHEAD, BPHEAD).
(q) For clarity, the previously named "Call Parameters"
and "Return Parameters" have been renamed "Input
Parameters" and "Output Parameters" respectively.
2.3 Differences between Version 2.00 and 1.00
(a) Bank Set Names may now be any Uppercase Alphabetic
Character (A-Z) rather than just C,E,R,S and T
(Version 2.00).
(b) Secondary YBOS Arrays (JW) are now totally
independent of the Primary YBOS Array
(COMMON/BCS/IW). Previously the Bank Name Table and
Hash Algorithm Table for all Arrays was held in IW.
There is only one exception to this; That is for the
YBOS Input/Output Subroutines. The Physical Record
Banks are always kept in /BCS/IW independent of the
array within which the Logical Record Banks appear
(Version 2.00)
(c) YBOS Arrays may be dumped to a Discfile and later
restored from the file. This is intended to allow
YBOS oriented histograms to be accessed by programs
other than the originating one. In addition,
individual Banks may be read from such Discfiles.
Refer to the section on Disc Based YBOS for details
(Version 2.00)
(d) Several Checkwords have been included in the YBOS
Header. These can be verified and hence provide some
measure of protection against corruption (Version
2.00).
(e) Several additional Subroutines are provided. These
mainly are involved in the Discfile accesses
mentioned previously and also in providing extra
printout and debugging facilities (Version 2.00).
3 INTRODUCTION
YBOS is a memory management system and data structure
that serves two main purposes:-
(a) It enables a hierarchical event structure to be set
up that allows rapid access to major subsections of
an event. This data structure is simple and compact,
introducing a minimal overhead in terms of space and
time. Subsections (called banks) may be manipulated
in the course of event processing, being extended,
deleted or created so as to allow a flexible
transition between the raw data and fully processed
events.
(b) It allows for modular programming while avoiding the
explosion in work space that sometimes results.
Portions of analysis programs may be written
independently while sharing a global work area. As
modules require work space they request it, releasing
it before passing on to further processing modules.
Basic data elements in the YBOS system are banks which
may be either named banks or work banks. Banks consist of a
header that identifies them and describes their size, and a
data area that contains the user's data. Banks are always an
integral number of Integer words long, although this does not
imply any restriction in the format of the user's data. All
indices to banks are Integer relative in the FORTRAN sense
(i.e. relative to 1).
In order to run YBOS, a computer must have an
architecture where an INTEGER has at least 32 bits of storage
and can contain at least 4 characters.
3.1 Named Banks
Named Banks are identified by a 4 character bank name and
a bank number. The bank name may be any 4 characters, there
being no formal restriction to alphanumeric names, but it is
highly recommended that names be declared such that they are
printable. The bank number must be a positive integer. A
named bank consists of two sections, a bank header and a data
area. The bank header is managed by YBOS and, except in rare
circumstances, should not be modified by the Application
Program, while the data area may be used to store the
particular data array desired by the application. Banks can
be of variable length, having, if so desired, no data area.
Once a bank has been created, the contents and length of the
data area may be modified by the application program.
A bank is located by its index within an array (the
particular array being defined by the user), and has both a
primary bank index that points to the bank header and a data
index that points to the start of the data area of the bank.
This data index will only be setup if the bank is created with
an extended bank header describing the internal format of the
data area of the bank. This description may be used to
perform automatic reformatting of banks when passing data
between computers with different machine architectures and
data representations.
Lists or sets of banks can be set up that may be
manipulated as a single entity. Thus the banks associated
with one bank set name may be added to or deleted from those
associated with another bank set name etc.
Named banks may be created, located, deleted, associated
with a set of banks, disassociated with a set of banks, and
both read from and written to permanent storage in a form
consistent with the logical and physical record structures
described in Appendix E.
Banks with the same name but different numbers are
chained together by pointers contained within the bank header
allowing rapid movement down such chains. Optionally a work
bank (see next section) may be created containing the indices
for all banks of a particular name allowing the bank indices
to be determined by direct table lookup.
When a named bank is deleted no automatic garbage
collection is performed, but subprograms exist to do this
explicitly. Garbage collection entails detecting any holes
resulting from deleted banks and shuffling existing banks
around to merge any non-contiguous holes together so as to
maximise the available free space. Note however, that
performing a named bank garbage collection may move some of
the remaining banks such that their indices change. It is the
application programs responsibility to relocate (determine the
new indices of) the remaining banks (unless a table lookup
work bank has been setup in which case this will be performed
automatically).
The detailed structure of a named bank is described in
Appendix A. An Application Programmer can choose to ignore
the detailed structure of the bank header, using subprogram
calls to locate elements of interest to him, or may directly
access the individual elements. The former approach is highly
recommended since it will cause the program to remain
compatible with any future versions of YBOS that might alter
the format of the bank header.
For the remainder of this document the bank name is
referred to as NAME, being either a Character*4 variable or 4
character long quoted text string (e.g. 'FRED'). The bank
number is denoted by NR, being a positive integer. The bank
index is denoted by IND and the data index is denoted by
DATIND.
3.2 Work Banks
Work banks are identified by an index only rather than by
name. They consist of a work bank header and user data area.
As in the case of named banks, the user data area can be of
variable length and may be modified after creation of the work
bank. A name may be associated with a work bank but its only
function is as a debugging aid in reading dumps, etc.
Work banks may only be created, deleted or modified.
None of the other functions (association with a set of banks,
Input/Output etc.) allowed for named banks are supported for
work banks. In particular, no detailed description of the
internal data structure of a work bank is supported. In
contrast to the case of named banks, an automatic garbage
collection will be performed automatically if necessary,
unless YBOS is explicitly set into a mode where any movement
of work banks is inhibited. Thus work banks may be relocated
transparently without the Application Program being informed
(except as noted above). The contents of the variable
containing the index of the bank will automatically be
modified if the work bank is moved through garbage collection
operations. Note however, that only the contents of this
single variable will be modified, not any other variables
that, through the use of assignment statements, might have
contents depending on this variable.
YBOS provides the facility for a work bank to be setup to
contain the indices to all the named banks having a particular
name. This allows the use of this direct table lookup in
locating thes banks. Such a work bank will be automatically
updated by YBOS if new named banks (with that name) are
created or if the banks are moved through explicit garbage
collection operations on the named bank storage area.
Also in contrast to named banks, a work bank can keep
track independently of the both total length and the data area
length of the bank. This allows the possibility of modifying
the length of the data area (within the limits determined by
the total length) without moving the work bank or creating
gaps that might get removed through garbage collection
operations. Thus a guaranteed data area maximum length may be
setup.
The detailed structure of a work bank is described in
Appendix B.
3.3 Comparison of Named Banks and Work Banks
In typical applications named banks will be used for all
long-living data (input, output, communication between program
modules) while work banks will be used for short-lived data
(intermediate results and communications inside a module).
Comparison of Named Banks and Work Banks
+----------------------------------------------------------+
| | |
| Named Banks | Work Banks |
| | |
+----------------------------------------------------------+
| | |
| Stored from the low index | Stored from the high index |
| part of each array | part of each array |
| | |
| No automatic garbage | Automatic garbage |
| collection | collection |
| | |
| No automatic update | Automatic update of |
| of pointer to Bank | pointer to Bank |
| | |
| Input/Output | No Input/Output |
| | |
| Extended Bank Header | Short Bank Header |
| with Bank Type allowing | No Machine Independence |
| Machine Independence | |
| | |
+----------------------------------------------------------+
3.4 Bank Storage
Both named and work banks may be stored in one of several
arrays, usually (but not necessarily) Common Blocks. There is
one basic array IW in Common Block /BCS/, and additional
arrays KW may be used if desired. Such arrays are totally
independent and self-contained. There are no
cross-connections between them except in one case. That is
for Sequential Input/Output where the primary YBOS array in
Common /BCS/ must always have been initialised irregardless of
which array the data to be read to or written from.
The detailed layout of a YBOS array is described in
detail in Appendix C.
3.5 Protected Named Banks
In order to allow some banks to be protected from being
dropped during normal cleanup operations, named banks whose
names begin with either a "+" character or "$" character
(e.g. '+DSK' or '$SAV') can only be dropped individually, not
in conjunction with a list of banks.
4 BANK TYPES
Version 3.00 of YBOS allows the Bank Header to be
extended to include the concept of a Bank Type. This
describes the internal data format of the Bank; whether the
data is Integer, Real*4 etc. or a mixture of datatypes. The
intention is to allow for the automatic conversion of
datawords for machines with different architectures and data
representations.
The implementation of the extended Bank Header to include
the Bank Type descriptors is backwards compatible such that
all existing YBOS routines and subroutines are unchanged.
Extra subprograms have been added to provide access to the
various sections of the Bank Header and to the data within a
Bank.
The detailed format of the extended bank header is given
in Appendix B.
4.1 Access to Data within Bank
Since the length of the extended Bank Header (including Bank
Type and Group Type words) can vary depending on whether the
Bank is Mono-Type or Mixed-Type, additional subprograms have
been added to YBOS to allow components of the Header to be
accessed, as well as the base Index of the Data. For the
remainder of this document, the Base Index of the Data is
given the name INDDAT, and is defined such that JW(INDDAT)
contains the first dataword in the Bank.
An example of how to access data within a Bank is given
in an Example in Appendix F. It is strongly recommended that
an Application Program follow this example.
4.2 Bank Types Parameter Definition File
An Include File defining the standard bank data types has
been setup. This is described in detail in Appendix B.
5 ALIAS BANK NAMES
A problem with YBOS (Versions 1.00 and 2.00) is that Bank
Names are limited to 4 alphanumeric characters. This is a
severe handicap in assigning a name having nmemonic value to a
YBOS Bank. In an attempt to overcome this problem, Version
3.00 allows an Alias Name to be defined for each Bank Name.
Internally YBOS uses just the 4 character basic Bank Name, but
the Application Programmer may declare an Alias Name of up to
32 alphanumeric characters in length. Subprograms have been
added that enable this declaration, and return the basic Bank
Name given the Alias Name. In addition, an extra routine has
been added to locate a Bank via its Alias Name. This just
combines the functions of the Name translation routine and the
normal Bank location routine and therefore has some extra
overhead involved. All other Named Bank access subprograms
described in the following sections utilise the basic 4
character Bank Name.
6 SETS OF BANKS
Sets of banks are defined by lists of bank names which
are associated with a single character. This set may be
handled as a single entity, being read from permanent storage,
expanded or reduced in scope, copied or appended to other sets
or dropped by single subprogram calls. Subprograms exist for
simple bank set manipulations and inquiries, as well as for
more complex input/ouput operations with bank sets. The
typical environment for the use of such sets is one where an
event record is read, analyzed in such a way as to add extra
information (in the form of banks) to the event (or perhaps
delete redundant banks), and is then written to further
permanent storage.
26 different sets may be defined, each being specified by
a single uppercase letter (A-Z). The "*" is used to indicate
all bank names.
7 SUBSETS OF BANKS
Subsets of banks are defined by lists of bank numbers
which are associated with a single bank name. One Bank Subset
is allowed per bank name. The members of a subset for a
particular name are stored in a work bank maintained
internally by YBOS. Bank subsets are used in conjunction with
sequential input/output routines in which only banks that are
subset members for the specified names are read/written.
Subprograms (described latter in this document) have been
added for simple bank subset manipulations, such as add and
deleteing banks from subsets, as well as for more complicated
input/output.
8 USER ACCESSIBLE YBOS LOCATIONS
A common situation is that one module in a program
creates a work bank (e.g. containing the indices of a set of
named banks). YBOS only keeps track of the location
containing the index to the work bank, updating its contents
if the work bank is moved due to a garbage collection etc.
Consider the following code fragment:-
STATUS = BLINK(JW,NAME,NRMAX,IND)
JND = IND
STATUS = WGARB
IND contains the Index to the created Work Bank, and the
contents of IND will be modified by YBOS if the call to WGARB
moves the work bank. However, JND will only contain the old
index to the work bank and its contents will not be
automatically updated if the work bank has moved through the
call to WGARB. Attempting to access the work bank via JND is
therefore unreliable and dangerous. Version 3.00 of YBOS
allows work banks to be explicitly locked such that they
cannot be moved by garbage collections.
In order for several subroutines to access the same work
bank, the pointer to the work bank (IND in the above example)
must either be transferred as a parameter of each subroutine
call, or a Common Block setup, within which IND resides.
While the former technique (parameter to subroutine call) is
preferred, there are situations where it is inappropriate.
Version 3.00 allows cooperating modules to utilise locations
within YBOS itself for performing the transfer of these
pointers between themselves. The mechanism is for a
reservation to be attempted of a User accessible location. If
this reservation is successful, then the location may be used;
if unsuccessful, then another location should be tried. Other
subroutines can access, but should not modify, the reserved
locations. Note, however, that there is no formal protection
to prevent illegal access to reserved locations.
The use of the User accessible Locations is not limited
only to the above function, but can be for more general
communications between program modules.
32 such User accessible Locations are defined within
YBOS.
9 YBOS OPERATING MODES
Version 3.00 introduces the concept of YBOS operating modes.
Various modes of operation for YBOS may be setup on request
through the use of subprogram calls. The following modes may
be affected:-
(a) Work Bank Locking. Normally YBOS has the freedom to
move Work Banks so as to optimise the available
space. This movement takes place transparently to
the Application Program and can give rise to problems
in certain instances. To prevent this, the
Application Program may "Lock" all Work Banks for the
duration of a critical piece of code, and then later
Unlock them. Note that the default is for work banks
to be unlocked.
(b) Error Message Reporting. The YBOS error level may be
specified so as to control the printing of error
messages. The error level may take on the following
values:-
Level 0 Disable all automatic error message
reporting
Level 1 Print error message
Level 2 Print short error message and dump YBOS
array (omitting the contents of banks).
Level 3 Print error message and perform full dump of
YBOS array, including bank contents.
The default is Level 1.
10 SUBPROGRAMS, FUNCTION VALUES AND ERROR CODES
The following sections describe in detail the
Programmer's Interface to the YBOS Package. All subprograms
may be accessed as Integer Functions where the Function Value
is either:-
(a) An Error Code as described below.
(b) A Bank Index.
In both cases the Error Code is made accessible to the caller
in the 2nd element of the appropriate YBOS array, and in
addition, subprogram BERROR may be used to return the Error
Code from the previous YBOS call.
The caller should refer to the Error Code by name rather
than number, these names being defined in an "Include" file to
be inserted in the header section of each program module
(subroutine or function) that accesses YBOS subprograms. For
VAX installations these Error Code Parameter Definitions
reside in file:-
YBOS$Library:ErrCod.Inc
For non-VAX installations consult your local YBOS expert.
The following Error Codes are defined:-
YESUCC Successful Completion
YECORR YBOS array header corrupted
YEILUN Illegal Logical Unit No.
YEIOUS Using Input/Ouput for Output/Input
YEEOF End-of-File Detected on Input
YELROF Logical Record too big for Space
YELRLN Logical Record Length Error
YEPRBD Bad Physical Record
YEBKOS Bank out of Sequence on Input
YEBKXD Bank Length exceeds Data on Input
YEBKNG Negative Bank Length
YEBKTS Bank Length too small
YEBSIL Illegal Bank Set Name
YELRZL Attempting to write zero length record
YEILOP Illegal Operand
YEOVRF Insufficient Space (Overflow)
YEBKSP Insufficient Space to extend Bank
YENOBK Non-existent Bank Name
YENONR Non-existent Bank Number
YEMANY Too many Bank Names
YEUSAR Using Basic Array with BOSAR
YEILUS Illegal User Location No.
YERSUS User Location No. already reserved
YENRUS User Location No. not reserved
YEDUPB Attempt to create Duplicate Bank
YEWLOK Work Banks Locked
YEILLA Illegal Alias Name
YEDUPA Duplicate Alias Name
YEBKOV Insufficient space to create bank
YEBTIL Unrecognized bank Id
YEGRIL Unrecognized group Id
YECHKF Internal consistancy check failed
YESTOF Stack limit exceeded
YENOTA Data not alligned
YEBNKP Bank padded with extra words
YEINDX Corrupted bank index warning...BVALID
YEBDCH Corrupted bank chain...BVALID
YEBDBK Corrupted bank header word...BVALID
YEBDLU Corrupted Blink lookup work bank...BVALID
YEBDBS Corrupted binnary search bank...BVALID
YEDKOP Unable to open diskfile
YEDKWT Unable to write to diskfile
YEDKRD Unable to read diskfile
YERSTF Diskfile dictionary restore failed,
rebuild to follow
The relationship between these Error Codes as Parameters and
the actual Integer Numbers represented by them may be
installation dependent and, in particular, may not be
backwards compatible. Application Programmers are strongly
recommended to use the Parameters rather than Numbers when
testing Error Codes.
next page