What do you do when you need to convert back and forth between EBCDIC and ASCII?
When dealing with systems other than the IBM i or communicating with languages other than RPG, you will need to use a common language to communicate. The native language of RPG is EBCDIC, and most non-IBM systems and languages work with the ASCII code set, so you will need to convert the characters going out of and coming into RPG. This is where the Coded Character Set Identifier (CCSID) value comes into play when you see it specified as a parameter of an API.
The CCSID is not a set of character sets; it is a set of identifiers to refer to character sets. This allows you to work with mixed byte-lengths and encoding by using an identifier that refers to an existing set of characters, such as EBCDIC and ASCII.
Some of the CCSID codes are interchangeable, like ASCII and ISO-8859, because the new code set was actually developed to be backward-compatible with the existing standard. Imagine that.... When the ISO-8859 standard was created, it repeated the complete set of the 128 ASCII characters and added another 128 characters after that. UTF-8 did the same thing to support ISO-8859 and extended it even further. UTF-16 did the same thing, but now we are talking about increasing the storage size to two bytes, so it's not physically equivalent.
Here is a list of some commonly used codes:
| Encoding | CCSID |
| EBCDIC | 37 |
| ASCII | 367 |
| ISO-8859-1 | 819 |
| UTF-8 | 1208 |
You can find the complete list of CCSIDs here.
The iconv() API converts from one CCSID to another. There is a CCSID for the EBCDIC code set, and there is also a CCSID for the ASCII code set. So you can use the iconv() API to convert between EBCDIC and ASCII.
Using iconv() is a three-step process:
1. Open the conversion descriptor.
2. Convert a buffer of characters using the conversion descriptor.
3. Close the conversion descriptor.
The QDCXLATE API is an alternative to using the iconv() API. The QDCXLATE API uses tables to convert from one character set to the other. The most recent tables used to support EBCDIC and ASCII are QTCPEBC and QTCPASC, respectively. This may be sufficient for simple cases, and you can also create custom tables for use with the QDCXLATE API. The iconv() API may be more complicated than QDCXLATE, but it is more flexible and updated.
My objective for this article is to convert between CCSIDs using procedures that will encapsulate the APIs to provide simplified code. The following code illustrates what the final code will look like to convert some EBCDIC bytes into ASCII bytes after we have built some custom procedures that can easily be dropped into a service program.
/free
// EBCDIC to UTF-8 (ASCII Superset)
toCCSID = 1208;
ebcdicString = 'AS/400';
cd = Converter_open(toCCSID);
asciiString = Converter_convert(cd: ebcdicString);
Converter_close(cd);
/end-free
If you were to debug this program and place a breakpoint immediately after the Converter_convert procedure executes, you would see the following results:
eval ebcdicString = 'AS/400 '
eval ebcdicString:x = C1E261F4 F0F04040 4040....
eval asciiString = ' ë▯▯▯▯ '
eval asciiString:x = 41532F34 30304040 4040....
As you can see here, there is minimal complexity. I was able to remove the need for pointers and complex data structures, with the exception of the conversion descriptor implementation of the iconv_t data structure. I contemplated putting a global structure in the service program, but I changed my mind because I wanted to support multiple construction descriptors in the application. Another alternative was to open and close the conversion descriptor within every conversion call, but I wanted to reduce the overhead by opening and closing the conversion descriptor only once. There's always the balance of simplicity versus speed, and you may choose a different alternative, but this code seems simple enough.
You may notice that the Converter_open procedure is using only one CCSID in this example. That's because it is assuming that you're converting from EBCDIC and is using the default system value that is indicated by setting the value of zero. If you were going in the opposite direction--ASCII to EBCDIC--you would specify both the to and from CCSID values as follows:
/free
// UTF-8 to EBCDIC
toCCSID = 0;
fromCCSID = 1208;
cd = Converter_open(toCCSID: fromCCSID);
ebcdicString = Converter_convert(cd: asciiString);
Converter_close(cd);
/end-free
You can download the complete code sample here.
The APIs
Now that the objective has been defined, let's dig into the APIs used to perform the conversion.
QtqIconvOpen: Open the Conversion Descriptor
The QtqIconvOpen API will provide you with a conversion descriptor that the iconv API can use to convert between CCSIDs. We will be converting between EBCDIC and ASCII CCSIDs. When specifying EBCDIC, you can use the value of 37 or you can use the value of 0 to specify the currently used CCSID.
Note: There is an alternative API for the QtqIconvOpen API called iconv_open() that has the same functionality. The only difference is in the interface, which uses characters instead of the QtqCode_T data structure.
These are the parameters you'll use:
- toCode () specifies the destination CCSID of the conversion results. Attribute: QtqCode_t data structure
- fromCode specifies the source CCSID of the characters to be converted. Attribute: QtqCode_t data structure
The QtqCode_t data structure is defined in the QSYSINC/QRPGLESRC, QTQICONV source file.
DQtqCode_t DS
D QTQCCSID 1 4B 0
D QTQCA 5 8B 0
D QTQSA 9 12B 0
D QTQSA00 13 16B 0
D QTQLO 17 20B 0
D QTQMEO 21 24B 0
D QTQERVED02 25 32
Return: The Conversion Descriptor in an iconv_t Data Structure
The iconv_t data structure that is returned by the QtqIconvOpen API is defined in the QSYSINC/QRPGLESRC, ICONV source file.
Diconv_t DS
D ICORV 1 4B 0
D ICOC 5 52B 0 DIM(00012)
This is the QtqIconvOpen prototype:
D QtqIconvOpen PR ExtProc('QtqIconvOpen')
D like(iconv_t)
D argToCCSID like(QtqCode_t) const
D argFromCCSID like(QtqCode_t) const
iconv: Convert a Buffer of Characters Using the Conversion Descriptor
The iconv API converts a set of characters from one CCSID to another using the conversion descriptor. The original characters to be converted will be placed into the input buffer, and the results will be returned in the output buffer.
| Parameters | Input/Output | Type |
| Conversion Descriptor | Input | iconv_t |
| Pointer to Input Buffer | Input | Char(*) |
| Input Buffer Bytes Left | Input | Integer(10) |
| Pointer to Output Buffer | Output | Char(*) |
| Output Buffer Bytes Left | Output | Integer(10) |
Here's the iconv prototype:
D* Convert CCSID from Input Buffer to Output Buffer
D iconv PR ExtProc('iconv')
D argConvDesc like(iconv_t) value
D argInBuffer *
D argInBytes 10I 0
D argOutBuffer *
D argOutBytes 10I 0
iconv_close: Closing the Conversion Descriptor
The iconv_close API closes the conversion descriptor.
Here's the iconv_close prototype:
D* Close the Conversion Descriptor
D iconv_close PR 10I 0 ExtProc('iconv_close')
D argConvDesc like(iconv_t) VALUE
Creating the Custom Procedures
Opening the Conversion Descriptor
*-----------------------------------------------------------------
* Converter_open: Opens the Conversion Descriptor for iconv
*-----------------------------------------------------------------
P Converter_open...
P B export
D Converter_open...
D PI likeDs(iconv_t)
D argToCCSID 10I 0
D argFromCCSID 10I 0 options(*nopass)
D* Local Variables
D from DS likeDs(QtqCode_t)
D to DS likeDs(QtqCode_t)
D cd DS likeDs(iconv_t)
D********************************************************************
/free
// Set the target CCSID
to = *ALLx'00';
to.QTQCCSID = argToCCSID;
to.QTQSA00 = 1;
// If Specified, Set the From CCSID
from = *ALLx'00';
if %PARMS < 2;
from.QTQCCSID = 0;
else;
from.QTQCCSID = argFromCCSID;
endif;
from.QTQSA00 = 1;
// If Specified, Set the From CCSID
cd = QtqIconvOpen(to: from);
if (cd.ICORV < *zeros);
// FAILURE
else;
// SUCCESS
endif;
return cd;
/end-free
P E
Converting Between CCSIDs Specified Within the Conversion Descriptor
*-----------------------------------------------------------------
* Converter_convert: Converts the CCSIDs of Conversion Descriptor
*-----------------------------------------------------------------
P Converter_convert...
P B EXPORT
D Converter_convert...
D PI 65535A
D argCd likeDs(iconv_t)
D argInString 65535A const varying
D********************************************************************
D inBuf S 65535A
D inBufPtr S *
D inBufSize S 10I 0
D outBuf S 65535A
D outBufPtr S *
D outBufSize S 10I 0
D********************************************************************
/free
inBuf = argInString;
inBufPtr = %addr(inBuf);
outBufPtr = %addr(outBuf);
// Size is only the initial value
// Number of Bytes Left is the true value
inBufSize = %len(%trimr(inBuf));
outBufSize = %size(outBuf);
Thomas Snyder has a diverse spectrum of programming experience encompassing IBM technologies, open source, Apple, and Microsoft and using these technologies with applications on the server, on the web, or on mobile devices.
Tom has more than 20 years' experience as a software developer in various environments, primarily in RPG, Java, C#, and PHP. He holds certifications in Java from Sun and PHP from Zend. Prior to software development, Tom worked as a hardware engineer at Intel. He is a proud United States Naval Veteran Submariner who served aboard the USS Whale SSN638 submarine.
Tom is the bestselling author of Advanced, Integrated RPG, which covers the latest programming techniques for RPG ILE and Java to use open-source technologies. His latest book, co-written with Vedish Shah, is Extract, Transform, and Load with SQL Server Integration Services.
Originally from and currently residing in Scranton, Pennsylvania, Tom is currently involved in a mobile application startup company, JoltRabbit LLC.
MC Press books written by Thomas Snyder available now on the MC Press Bookstore.
 |
||
 |
|
Advanced, Integrated RPG See how to take advantage of the latest technologies from within existing RPG applications. List Price $79.95 Now On Sale
|
|
||
 |
|
Extract, Transform, and Load with SQL Server Integration Services Learn how to implement Microsoft’s SQL Server Integration Services for business applications. List Price $79.95
Now On Sale
|
More than ever, there is a demand for IT to deliver innovation. Your IBM i has been an essential part of your business operations for years. However, your organization may struggle to maintain the current system and implement new projects. The thousands of customers we've worked with and surveyed state that expectations regarding the digital footprint and vision of the company are not aligned with the current IT environment.
TRY the one package that solves all your document design and printing challenges on all your platforms. Produce bar code labels, electronic forms, ad hoc reports, and RFID tags – without programming! MarkMagic is the only document design and print solution that combines report writing, WYSIWYG label and forms design, and conditional printing in one integrated product. Make sure your data survives when catastrophe hits. Request your trial now! Request Now.
Forms of ransomware has been around for over 30 years, and with more and more organizations suffering attacks each year, it continues to endure. What has made ransomware such a durable threat and what is the best way to combat it? In order to prevent ransomware, organizations must first understand how it works.
Disaster protection is vital to every business. Yet, it often consists of patched together procedures that are prone to error. From automatic backups to data encryption to media management, Robot automates the routine (yet often complex) tasks of iSeries backup and recovery, saving you time and money and making the process safer and more reliable. Automate your backups with the Robot Backup and Recovery Solution. Key features include:
Business users want new applications now. Market and regulatory pressures require faster application updates and delivery into production. Your IBM i developers may be approaching retirement, and you see no sure way to fill their positions with experienced developers. In addition, you may be caught between maintaining your existing applications and the uncertainty of moving to something new.
IT managers hoping to find new IBM i talent are discovering that the pool of experienced RPG programmers and operators or administrators with intimate knowledge of the operating system and the applications that run on it is small. This begs the question: How will you manage the platform that supports such a big part of your business? This guide offers strategies and software suggestions to help you plan IT staffing and resources and smooth the transition after your AS/400 talent retires. Read on to learn:
LATEST COMMENTS
MC Press Online