For years now, I've been using iconv() to convert between different code pages, or CCSIDs. This routine alters data by creating a conversion environment and then mapping the bit patterns of your data to the corresponding bit patterns of another character set.

Converting from U.S. English to Italian (for example) does not translate the words, but rather translates the characters. A more logical example might be converting between North American EBCDIC and so-called PC ASCII. To do this, you would need to convert from CCSID 37 to CCSID 819.

The iconv(), iconv_open(), and iconv_close() APIs are among the most complex APIs for people to start using. Many people have limited their API usage to QCMDEXC and the occasional QCLSCAN call. Jumping right into iconv() after using only these two APIs or other simple APIs such as QUSLFLD can be vexing at best.

The complicated part of iconv() comes in two parts:

1. You have to create a conversion environment using the open API and then close the conversion environment using the close API.
2. You call iconv() to do the conversion, and it updates all the input values to reflect the data that it did not convert. The input text length is reduced to the number of characters yet to be converted (often this means zero is returned), and the output buffer length is reduced as well. In addition, pointers need to be passed to the API; these pointers are adjusted to point to the next position in the output buffer for the conversion. If everything has been converted, this means they point to the end of the data. This causes lots of confusion with first-time API programmers.

Because of this, many programmers fall back on the old System/38 APIs QDCXLATE or QTBXLATE (synonyms). See my article on this topic.

A problem occurs when using QDCXLATE, however. When the IBM-supplied conversion tables QASCII, QEBCDIC, and QTCPASC are used, the API produces invalid results. That is, if you convert from EBCDIC to ASCII, you do not get clean results.

These conversion tables are so old that even IBM doesn't remember what many of them were used for. The only thing I could confirm is that QASCII was built to support a 20+-year-old ASCII tape device on the System/38, long before CCSIDs were in place. These tables contain invalid data by today's standards and should never be used. This doesn’t mean the API should not be used; it simply means that using QDCXLATE to convert between these two fundamental character sets should be avoided.

As mentioned, the iconv() API comes in three parts: open, convert, and close. But there is a variation on the open APIs as well, which can add to the confusion.

• iconv_open() creates a conversion environment using the original conversion data structure. This data structure more complicated than the one used by QtqIconvOpen.
• iconv() converts data.
• iconv_close() destroys the conversion environment.
• QtqIconvOpen creates a conversion environment using a more RPG-friendly conversion data structure.

Most people use the iconv_open to create the conversion environment, but I prefer the QtqIconvOpen because it supports a more practical data structure. In addition, iconv_open() uses two different data structures for the from/to CCSIDs, which just adds to the confusion.

The QtqIconvOpen API uses the following data structure to create the conversion environment:

The only field we need to adjust when using this data structure is the CCSID subfield. We set the CCSID subfield to the original and target CCSIDs. Thus, we actually need two data structures; one for the original CCSID, and one for the target CCSID. So we need to code something like the following:

Since we have the ability to create multiple simultaneous conversion environments, iconv() uses a "handle," a data structure that controls the conversion efforts for calling iconv. This handle/data structure is defined below:

To create a conversion environment, we need to call QtqIconvOpen. The following prototype allows QtqIconvOpen to be called from RPG IV:

Calling QtqIconvOpen to establish/create the conversion environment to convert from the job's CCSID to ASCII used on the IFS, we would use the following code:

The source CCSID is 0, which indicates that the job's CCSID should be used. In the U.S. this is normally 37. In other countries, it will be whatever CCSID your job is using. But 0 is the current value to specify. The target CCSID is 819, which is often used for ASCII files stored on the IFS.

The next step is to call iconv() to actually perform a conversion. To do this, we need to prototype iconv, as follows:

This API uses a lot of pointers, and more and more stuff is requiring the use of them, so we should just accept it and move on. The iconv API is a little more complex with its use of pointers, however. It needs a pointer to a pointer. You can't simply pass in the address of a variable to convert; you need to pass the address of the address. Sound confusing? It is.

For example, if we want to convert the data in the field named MYDATA, we would need the following declarations:

The first line is the field we want to convert. The second field, pData, is a pointer to that MYDATA variable. But iconv needs a pointer to a pointer, so we need a second variable that is also a pointer containing the address of the first pointer. Geez! The ppData variable accomplishes this for us.

To provide a more practical example, the pData variable is typically set in the Calc specifications using the %ADDR built-in function. The following example illustrates calling iconv to convert the content of MYDATA by using the conversion environment created in the QtqIconvOpen example above:

After this API is called, the data in MYDATA is converted to CCSID 819. If we are finished with the conversion, we should close the conversion environment. To do that, we need to call iconv_close. The prototype for this API follows:

To close/destroy the conversion environment, simply call iconv_close and pass to it the handle from the previous iconv_open or QtqIconvOpen API, as follows:
C callp iconv_close(hConv)
C eval hConv = *ALLX'00'

I also like to clear the conversion handle just for good measure. That's what the second line of code in the above example is doing.

iSockets: OpenCvtCCSID, CvtCCSID, CloseCvtCCSID

In the free iSockets library, I've created simplified API wrappers for iconv. Prototypes for these subprocedures follow:

If you own RPG xTools, you can use the even easier XLATECCSID procedure, which allows you to convert a variable and specify the from/to CCSID values and avoid dealing with declaring all those data structures.

Whatever level you want—directly calling the APIs, using the iSockets helper functions, or calling RPG xTools' easyAPI subprocedures—converting between character sets can be easy and accurate with iconv.