23
Sat, Nov
1 New Articles

What You Should Know About the Format of Records Saved to a Save File, Part 2

RPG
Typography
  • Smaller Small Medium Big Bigger
  • Default Helvetica Segoe Georgia Times

 

Do you need to retrieve data from variable-length fields of deleted records?

 

The data of variable-length fields of records deleted from a file can be retrieved provided that the inserts and updates performed in the file after the records were deleted didn't affect the variable part of the file.

In the first part of this article, we saw that, when a physical file is saved to a save file, the fixed and the variable parts of every recordthat is, its records A and Bare stored separately in areas A and B of the save file. We also described the format of records A and B and saw that the data stored in area A of the save file could be retrieved for the deleted records.

In the second part of the article, we will see how the system manages the data of the variable part of the records and determine under which conditions this data can be retrieved for the deleted records.

Effect of File Operations on Records B

When a record is deleted from a file, the only field changed in its corresponding record B is RRN1, which is set to 0 (see section "Format of Records B" in the first part of the article). In addition, the space taken up by this record B is marked as available so that it can be reused to store data for another record of the file. This space is marked as available by adding an item containing its length, segment number, and segment offset to an array located in an area of the save file that we will call area 14. As the only change that needs to be done in area B when a record is deleted is to set field RRN1 to 0, the system can delay this change until another access to area B is requested.

When a record of a file is updated, the new values that are to be stored in area B are kept in the current record B if it's long enough to hold them. If the new values don't fit into this record B, the system looks for an available record B (that is, a record B whose length and location are stored in area 14) long enough for the new values. If such a record B is found, the system reuses it to store the new values. If it's not found, a new record B is added at the end of area B. Finally, if the original record B could not be used to store the new values, its field RRN1 is set to 0 and the space it takes up is marked as available.

For example, let's insert the two following records into physical file FILE2:

     A          R RFILE2

     A            FLD1           1A

     A            FLD2          30A            VARLEN

     A            FLD3          30A            VARLEN

     A            FLD4           1A

insert into QTEMP/FILE2 values

   ('1', 'AAA', '12345', '0'),

   ('2', 'BBB', '12345', '0')

The two corresponding records B are the following:

Area B (save file record 57):

   Bk___ DataBk   nnnn RRN   wk AccumLenk

   E EndOfRecord

                                                    RRN  From  To

*...+....1....+....2....+....3....+....4....+....5   57     1  50

     © u 4ze¦                           AAA12345

01000B0A2FA86000000000000000000000000000CCCFFFFF00

030834040495A0000100000001000000000103081111234500

                                        B1_B2___

                                nnnnw1w2        nn

....+....6....+....7....+....8....+....9....+....0         51 100

  Ë        BBB12345    Ë

00700000000CCCFFFFF0000700000000000000000000000000

01300020308222123450002300000000000000000000000000

           B1_B2___

nnEnnnnw1w2        nnnnE

In the following update, the new values for fields FLD2 and FLD3 fit into the current record B:

update QTEMP/FILE2

set FLD2 = 'NEWAAA', FLD3 = '7'

where FLD1 = '1'

Area B:

*...+....1....+....2....+....3....+....4....+....5

     © u 4ze¦                           NEWAAA75

01000B0A2FA86000000000000000000000000000DCECCCFF00

030834040495A0000100000001000000000106075561117500

....+....6....+....7....+....8....+....9....+....0

  Ë        BBB12345    Ë

00700000000CCCFFFFF0000700000000000000000000000000

01300020308222123450002300000000000000000000000000

When a record is deleted, the field RRN1 of its corresponding record B is set to 0 and the space taken up by this record B is marked as available:

delete from QTEMP/FILE2

where FLD1 = '1'

Area B:

*...+....1....+....2....+....3....+....4....+....5

     © u 4ze¦                           NEWAAA75

01000B0A2FA86000000000000000000000000000DCECCCFF00

030834040495A0000100000001000000000006075561117500

....+....6....+....7....+....8....+....9....+....0

  Ë        BBB12345    Ë

00700000000CCCFFFFF0000700000000000000000000000000

01300020308222123450002300000000000000000000000000

Area 14 (save file record 65):

   ll length   ss segment number   oooo segment offset

*...+....1....+....2....+....3....+....4....+....5

     m u 4ze¦

0100090A2FA860000000000000000000010000020000000000

040834040495A0000100000000000000050100000000000000

                                llssoooo

In the next case, the new values for the record don't fit into its record B and the only record B available is not long enough for the new values, so a new record B is added at the end of area B:

update QTEMP/FILE2

set FLD2 = 'NEWBBB'

where FLD1 = '2'

Area B:

*...+....1....+....2....+....3....+....4....+....5

     © u 4ze¦                           NEWAAA75

01000B0A2FA86000000000000000000000000000DCECCCFF00

030834040495A0000100000001000000000006075561117500

....+....6....+....7....+....8....+....9....+....0

  Ë        BBB12345    Ë        NEWBBB12345    Ë

00700000000CCCFFFFF0000700000000DCECCCFFFFF0000700

0130000030822212345000230002060B556222123450002300

Area 14:

*...+....1....+....2....+....3....+....4....+....5

     m u 4ze¦

0100090A2FA860000000000000000000010000020100000300

040834040495A0000100000000000000050100000501000500

Let's delete the record we have just updated:

delete from QTEMP/FILE2

where FLD1 = '2'

Area B:

*...+....1....+....2....+....3....+....4....+....5

     © u 4ze¦                           NEWAAA75

01000B0A2FA86000000000000000000000000000DCECCCFF00

030834040495A0000100000001000000000006075561117500

....+....6....+....7....+....8....+....9....+....0

  Ë        BBB12345    Ë        NEWBBB12345    Ë

00700000000CCCFFFFF0000700000000DCECCCFFFFF0000700

0130000030822212345000230000060B556222123450002300

Area 14:

*...+....1....+....2....+....3....+....4....+....5

     m u 4ze¦

0100090A2FA860000000000000000000010000020100000301

040834040495A0000100000000000000050100000501000508

....+....6....+....7....+....8....+....9....+....0

     [

00000400000000000000000000000000000000000000000000

01000A00000000000000000000000000000000000000000000

When we insert the following record, the system reuses an available record B to store the fields FLD2 and FLD3. Note that the item in area 14 corresponding to this record B is initialized to 0.

insert into QTEMP/FILE2 values

('3', 'CCC', '123456', '0')

Area B:

*...+....1....+....2....+....3....+....4....+....5

     © u 4ze¦                           NEWAAA75

01000B0A2FA86000000000000000000000000000DCECCCFF00

030834040495A0000100000001000000000006075561117500

....+....6....+....7....+....8....+....9....+....0

  Ë        BBB12345    Ë        CCC12345645    Ë

00700000000CCCFFFFF0000700000000CCCFFFFFFFF0000700

01300000308222123450002300030309333123456450003300

Area 14:

*...+....1....+....2....+....3....+....4....+....5

     m u 4ze¦

0100090A2FA860000000000000000000010000020100000300

040834040495A0000100000000000000050100000501000500

....+....6....+....7....+....8....+....9....+....0

00000000000000000000000000000000000000000000000000

00000000000000000000000000000000000000000000000000

Area 14

Area 14, as it happens for areas A and B, consists of the positions 1 to 512 of a group of consecutive records of the save file. From its position 33, it stores an array of items 8 bytes long containing the length and location of records B whose space has become available and can be reused by the system. Figure 1 shows the format of these items:

 082113MaciasFigure1

Figure 1: This is the format of an item of area 14.

When an available record B is reused by the system to store data for a record inserted or updated in the file, its corresponding item in area 14 is initialized to 0.

To identify the record of the save file where area 14 begins, we'll use the fact that positions 7 to 18 of every segment of every area of a given member are the same. Beginning at the record of the save file that follows the first record of area B, we will look for a record of the save file whose position 2 is equal to x14 and whose positions 7 to 18 are equal to positions 7 to 18 of the first segment of area B.

The end of area 14 is determined by the beginning of the following area. This area begins at the following record of the save file whose position 2 is equal to x15 and whose positions 7 to 18 are equal to positions 7 to 18 of the first segment of area B.

Area 14 seems to have a maximum size, at least in many cases. Currently, this maximum size is 16MBthat is, one segmentso area 14 can store up to (16MB – 32 bytes) / 8 = 2,097,148 items. If this number of items is reached, the system will ignore the requests to store new items in area 14 until there is some free spacethat is, until some of the available records B are reused by the system and their corresponding items in area 14 are initialized to 0.

Retrieving Data from Records B

The record B that an item of area 14 points to can be found in the record of the save file and at the position that is shown in Figure 2:

082113MaciasFigure2

Figure 2: These are the record of the save file and the position where record B begins.

Field RRN2 of record B allows us to identify the corresponding record of the physical file and the corresponding record A in area A. Field Status of this record A will tell us whether the record of the physical file was deleted or not.

As we saw in section "Effect of File Operations on Records B", there can be several records B containing data for the same record of the physical file, one of them corresponding possibly to the insert of the record in the file and the others to some of the updates of the record. Taking into account how the system manages records B, we can get to the conclusion that the record B containing the latest data is the longest one.

In order to retrieve the data of the fields stored in area B for a record R that has been deleted from a file, we need to be certain that the latest record Bthat is, the record B containing the latest data of record Ris present in area B. If some other record has been inserted or updated in the file after record R was deleted, and this insert or update required a record B, the system may have reused the latest record B of record R.

If the system did reuse the latest record B and there are another records B corresponding to record R, it would be incorrect to consider one of these records B as the latest one. For example, when record no. 3 was inserted into FILE2 in section "Effect of File Operations on Records B", the system reused the record B containing the latest data of record no. 2 ("NEWBBB", "12345") to store the data for record no. 3 ("CCC", "123456"). The data from the other record B corresponding to record no. 2 ("BBB", "12345") is not the data record no. 2 had when it was deleted.

The only case where we can know that the latest record B has not been reused by the system is when its field RRN1 is different from 0. As we saw in section "Effect of File Operations on Records B", this happens when a record has been deleted from the file and the system has delayed setting to 0 the field RRN1 of the corresponding record B.

As a consequence, for each insert or update requiring a record B that has been performed in a file after the records were deleted, there may be a deleted record whose data in area B cannot be retrieved or can be retrieved but is not the latest one.

Another limitation to the retrieval of data from area B for a deleted record comes from the fact that area 14 has a maximum size. If an item cannot be stored in area 14 because this area has reached its maximum size, the location of the record B contained in that item is lost.

Finally, we must take into account that the fields whose data can be retrieved from the latest record B are only those whose corresponding length in record A is equal to x8000. A length of a field different from x8000 implies that the latest data for that field was not stored in the corresponding record B (see VarFieldLen in section "Format of Records A" in the first part of the article).

GETDLTRCD Command

The Get Deleted Records (GETDLTRCD) command is an implementation of the concepts seen in this article. It can be downloaded from the MC Press web site in English and Spanish. I would be grateful for your feedback on the command, especially if some error is found.

 

Juan Macias

Juan Macias has a Degree in Mathematics (1988) and a Master in Data Management from the Universidad Complutense de Madrid (1989). He's a developer of RPG and COBOL applications mainly for finance and insurance companies. He can be contacted at This email address is being protected from spambots. You need JavaScript enabled to view it..

BLOG COMMENTS POWERED BY DISQUS

LATEST COMMENTS

Support MC Press Online

$

Book Reviews

Resource Center

  • SB Profound WC 5536 Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application. You can find Part 1 here. In Part 2 of our free Node.js Webinar Series, Brian May teaches you the different tooling options available for writing code, debugging, and using Git for version control. Brian will briefly discuss the different tools available, and demonstrate his preferred setup for Node development on IBM i or any platform. Attend this webinar to learn:

  • SB Profound WP 5539More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.

  • SB HelpSystems ROBOT Generic IBM announced the E1080 servers using the latest Power10 processor in September 2021. The most powerful processor from IBM to date, Power10 is designed to handle the demands of doing business in today’s high-tech atmosphere, including running cloud applications, supporting big data, and managing AI workloads. But what does Power10 mean for your data center? In this recorded webinar, IBMers Dan Sundt and Dylan Boday join IBM Power Champion Tom Huntington for a discussion on why Power10 technology is the right strategic investment if you run IBM i, AIX, or Linux. In this action-packed hour, Tom will share trends from the IBM i and AIX user communities while Dan and Dylan dive into the tech specs for key hardware, including:

  • Magic MarkTRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now!  Request Now.

  • SB HelpSystems ROBOT GenericForms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.

  • SB HelpSystems ROBOT GenericIT security is a top priority for businesses around the world, but most IBM i pros don’t know where to begin—and most cybersecurity experts don’t know IBM i. In this session, Robin Tatam explores the business impact of lax IBM i security, the top vulnerabilities putting IBM i at risk, and the steps you can take to protect your organization. If you’re looking to avoid unexpected downtime or corrupted data, you don’t want to miss this session.

  • SB HelpSystems ROBOT GenericCan you trust all of your users all of the time? A typical end user receives 16 malicious emails each month, but only 17 percent of these phishing campaigns are reported to IT. Once an attack is underway, most organizations won’t discover the breach until six months later. A staggering amount of damage can occur in that time. Despite these risks, 93 percent of organizations are leaving their IBM i systems vulnerable to cybercrime. In this on-demand webinar, IBM i security experts Robin Tatam and Sandi Moore will reveal:

  • FORTRA Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:

  • FORTRAManaging messages on your IBM i can be more than a full-time job if you have to do it manually. Messages need a response and resources must be monitored—often over multiple systems and across platforms. How can you be sure you won’t miss important system events? Automate your message center with the Robot Message Management Solution. Key features include:

  • FORTRAThe thought of printing, distributing, and storing iSeries reports manually may reduce you to tears. Paper and labor costs associated with report generation can spiral out of control. Mountains of paper threaten to swamp your files. Robot automates report bursting, distribution, bundling, and archiving, and offers secure, selective online report viewing. Manage your reports with the Robot Report Management Solution. Key features include:

  • FORTRAFor over 30 years, Robot has been a leader in systems management for IBM i. With batch job creation and scheduling at its core, the Robot Job Scheduling Solution reduces the opportunity for human error and helps you maintain service levels, automating even the biggest, most complex runbooks. Manage your job schedule with the Robot Job Scheduling Solution. Key features include:

  • LANSA Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.

  • LANSAWhen it comes to creating your business applications, there are hundreds of coding platforms and programming languages to choose from. These options range from very complex traditional programming languages to Low-Code platforms where sometimes no traditional coding experience is needed. Download our whitepaper, The Power of Writing Code in a Low-Code Solution, and:

  • LANSASupply Chain is becoming increasingly complex and unpredictable. From raw materials for manufacturing to food supply chains, the journey from source to production to delivery to consumers is marred with inefficiencies, manual processes, shortages, recalls, counterfeits, and scandals. In this webinar, we discuss how:

  • The MC Resource Centers bring you the widest selection of white papers, trial software, and on-demand webcasts for you to choose from. >> Review the list of White Papers, Trial Software or On-Demand Webcast at the MC Press Resource Center. >> Add the items to yru Cart and complet he checkout process and submit

  • Profound Logic Have you been wondering about Node.js? Our free Node.js Webinar Series takes you from total beginner to creating a fully-functional IBM i Node.js business application.

  • SB Profound WC 5536Join us for this hour-long webcast that will explore:

  • Fortra IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn: