This article needs additional citations for verification. (March 2007) |
In computer science, a record-oriented filesystem is a file system where data is stored as collections of records. This is in contrast to a byte-oriented filesystem, where the data is treated as an unformatted stream of bytes. There are several different possible record formats; the details vary depending on the particular system. In general the formats can be fixed-length or variable length, with different physical organizations or padding mechanisms; metadata may be associated with the file records to define the record length, or the data may be part of the record. Different access methods for records may be provided, for example records may be retrieved in sequential order, by key, or by record number.
Origin and characteristics
Record-oriented filesystems are frequently associated with mainframe operating systems, such as OS/360 and successors and DOS/360 and successors, and midrange operating systems, such as RSX-11 and VMS. However, they originated earlier in software such as Input/Output Control System (IOCS). Records, sometimes called logical records, are often written together in blocks, sometimes called physical records; this is the norm for direct access and tape devices, but files on unit record devices are normally unblocked, i.e., there is only one record per block.
Record-oriented filesystems can be supported on media other than direct access devices. A deck of punched cards can be considered a record-oriented file. A magnetic tape is an example of a medium that can support records of uniform length or variable length.
In a record file system, a programmer designs the records that may be used in a file. All application programs accessing the file, whether adding, reading, or updating records share an understanding of the design of the records. In DOS/360, OS/360, and their successors there is no restriction on the bit patterns composing the data record, i.e. there is no delimiter character; this is not always true in other software, e.g., certain record types for RCA File Control Processor (FCP) on the 301, 501, 601 and 3301.
The file comes into existence when a file create request is issued to the filesystem. Some information about the file may be included with the create request. This information may specify that the file has fixed-length records (all records are the same size) along with the size of the records. Alternatively, the specification may state that the records are of variable length, along with the maximum record length. Additional information including blocking factor, binary vs. text and the maximum number of records may be specified.
It may be permitted to read only the beginning of a record; the next sequential read returns the next collection of data (record) that the writer intended to be grouped together. It may also be permitted to write only the beginning of a record. In these cases, the record is padded with binary zeros or with spaces, depending on whether the file is recognized as a binary file or a text file.
Some operating systems require that library routines specific to the record format be included in the program. This means that a program originally expected to read a variable length record file cannot read a fixed length file. These operating systems must provide file system utilities for converting files between one format and another. This means copying the file (which requires additional storage space, time and coordination) may be necessary.
Other operating systems include various routines and associate the appropriate routine, based on the file organization, at execution time.
In either case, significant amounts of code to manage records must be provided in protected routines to ensure file integrity.
An alternative to a Record-oriented file is a stream file, in which the file system treats a file as an unstructured sequence of bytes. The applications may, but need not, impose a record structure. This approach significantly reduces the size and complexity of the library and reduces the number of utilities required to maintain files.
A common application convention for text files represented as streams is to use a new line delimiter to separate or terminate records, commonly CR, CRLF or LF. Unfortunately, the CPU time required to parse for the record delimiter is significant and the exclusion of the record delimiter pattern from the data is frequently undesirable.
An alternate convention is to include a length field in each record. The writer application is responsible for imposing any record structure and the reader application is responsible for separating out the records.
Advantages and costs
This section does not cite any sources. (September 2023) |
A record-oriented file has several advantages. After a program writes a collection of data as a record the program that reads that record has the understanding of those data as a collection. Often a file will contain several related records in sequence; after the program reads the beginning of the sequence, the next sequential read returns the next collection of data (record) that the writer intended to be grouped together. Another advantage is that the record has a length and there is usually no restriction on the bit patterns composing the data record, i.e. there is no delimiter character.
There is usually a cost associated with record-oriented files. For fixed-length records, some records may have unused space, while for variable-length records the delimiter or length field takes up space. Variable-length blocks may have overhead due to delimiters or length fields. In addition, there is overhead imposed by the device. On a magnetic tape overhead typically takes the form of an inter-record gap. On a direct access device with fixed-length sectors, there may be unused space in the last sector of a block. On a direct access device with variable-length physical records, that overhead typically takes the form of metadata and inter-record gaps.
On a file composed of varying-length records a maximum record length is defined to determine the size of the length metadata associated with each record.
A major advantage of record-oriented file systems is that they abstract files kept on paper in earlier times. A record might contain data associated with a particular, e.g., building, contact, employee, part, venue.
A second motivator for the idea of record orientation is that it is in some sense the more natural orientation for persistent storage on a non-volatile but slow physical storage device. Most physical storage devices can communicate only in units of a block. Significant portions of modern operating system kernels and associated device drivers are devoted to hiding the naturally structured and delimited (and in some sense a block is just a physical record) nature of physical storage devices. It is not coincidental that record-oriented file systems arose earlier in the history of computing than byte-stream oriented file systems, when the capabilities for abstraction were far less.
See also
- Stream (computing)
- Data set (IBM mainframe)
- Files-11
- CMS file system
- ISAM/VSAM
- Distributed Data Management Architecture (DDM)
- Record-oriented file (DDM)
References
- z/OS DFSMS Using Data Sets Version 2 Release 3 (PDF), October 2, 2018, SC23-6855-30
- Reference Manual, IBM 709/7090 Input/output Control System (PDF). IBM. p. 3. C28-6100-2. Retrieved Sep 12, 2020.
This article needs additional citations for verification Please help improve this article by adding citations to reliable sources Unsourced material may be challenged and removed Find sources Record oriented filesystem news newspapers books scholar JSTOR March 2007 Learn how and when to remove this message In computer science a record oriented filesystem is a file system where data is stored as collections of records This is in contrast to a byte oriented filesystem where the data is treated as an unformatted stream of bytes There are several different possible record formats the details vary depending on the particular system In general the formats can be fixed length or variable length with different physical organizations or padding mechanisms metadata may be associated with the file records to define the record length or the data may be part of the record Different access methods for records may be provided for example records may be retrieved in sequential order by key or by record number Origin and characteristicsRecord oriented filesystems are frequently associated with mainframe operating systems such as OS 360 and successors and DOS 360 and successors and midrange operating systems such as RSX 11 and VMS However they originated earlier in software such as Input Output Control System IOCS Records sometimes called logical records are often written together in blocks sometimes called physical records this is the norm for direct access and tape devices but files on unit record devices are normally unblocked i e there is only one record per block Record oriented filesystems can be supported on media other than direct access devices A deck of punched cards can be considered a record oriented file A magnetic tape is an example of a medium that can support records of uniform length or variable length In a record file system a programmer designs the records that may be used in a file All application programs accessing the file whether adding reading or updating records share an understanding of the design of the records In DOS 360 OS 360 and their successors there is no restriction on the bit patterns composing the data record i e there is no delimiter character this is not always true in other software e g certain record types for RCA File Control Processor FCP on the 301 501 601 and 3301 The file comes into existence when a file create request is issued to the filesystem Some information about the file may be included with the create request This information may specify that the file has fixed length records all records are the same size along with the size of the records Alternatively the specification may state that the records are of variable length along with the maximum record length Additional information including blocking factor binary vs text and the maximum number of records may be specified It may be permitted to read only the beginning of a record the next sequential read returns the next collection of data record that the writer intended to be grouped together It may also be permitted to write only the beginning of a record In these cases the record is padded with binary zeros or with spaces depending on whether the file is recognized as a binary file or a text file Some operating systems require that library routines specific to the record format be included in the program This means that a program originally expected to read a variable length record file cannot read a fixed length file These operating systems must provide file system utilities for converting files between one format and another This means copying the file which requires additional storage space time and coordination may be necessary Other operating systems include various routines and associate the appropriate routine based on the file organization at execution time In either case significant amounts of code to manage records must be provided in protected routines to ensure file integrity An alternative to a Record oriented file is a stream file in which the file system treats a file as an unstructured sequence of bytes The applications may but need not impose a record structure This approach significantly reduces the size and complexity of the library and reduces the number of utilities required to maintain files A common application convention for text files represented as streams is to use a new line delimiter to separate or terminate records commonly CR CRLF or LF Unfortunately the CPU time required to parse for the record delimiter is significant and the exclusion of the record delimiter pattern from the data is frequently undesirable An alternate convention is to include a length field in each record The writer application is responsible for imposing any record structure and the reader application is responsible for separating out the records Advantages and costsThis section does not cite any sources Please help improve this section by adding citations to reliable sources Unsourced material may be challenged and removed September 2023 Learn how and when to remove this message A record oriented file has several advantages After a program writes a collection of data as a record the program that reads that record has the understanding of those data as a collection Often a file will contain several related records in sequence after the program reads the beginning of the sequence the next sequential read returns the next collection of data record that the writer intended to be grouped together Another advantage is that the record has a length and there is usually no restriction on the bit patterns composing the data record i e there is no delimiter character There is usually a cost associated with record oriented files For fixed length records some records may have unused space while for variable length records the delimiter or length field takes up space Variable length blocks may have overhead due to delimiters or length fields In addition there is overhead imposed by the device On a magnetic tape overhead typically takes the form of an inter record gap On a direct access device with fixed length sectors there may be unused space in the last sector of a block On a direct access device with variable length physical records that overhead typically takes the form of metadata and inter record gaps On a file composed of varying length records a maximum record length is defined to determine the size of the length metadata associated with each record A major advantage of record oriented file systems is that they abstract files kept on paper in earlier times A record might contain data associated with a particular e g building contact employee part venue A second motivator for the idea of record orientation is that it is in some sense the more natural orientation for persistent storage on a non volatile but slow physical storage device Most physical storage devices can communicate only in units of a block Significant portions of modern operating system kernels and associated device drivers are devoted to hiding the naturally structured and delimited and in some sense a block is just a physical record nature of physical storage devices It is not coincidental that record oriented file systems arose earlier in the history of computing than byte stream oriented file systems when the capabilities for abstraction were far less See alsoStream computing Data set IBM mainframe Files 11 CMS file system ISAM VSAM Distributed Data Management Architecture DDM Record oriented file DDM Referencesz OS DFSMS Using Data Sets Version 2 Release 3 PDF October 2 2018 SC23 6855 30 Reference Manual IBM 709 7090 Input output Control System PDF IBM p 3 C28 6100 2 Retrieved Sep 12 2020
