NAME

ShiftJIS::JISX0208::MapUTF - transcode between Shift_JIS (specified in Appendix 1 of JIS X0208:1997) and Unicode

SYNOPSIS

    use ShiftJIS::JISX0208::MapUTF;

    $utf8_string = shiftjis_to_utf8($sjis_string);
    $sjis_string = utf8_to_shiftjis($utf8_string);

DESCRIPTION

Shift_JIS table comprises 7070 characters:

    JIS X 0201 single-byte characters (191 characters),
    JIS X 0208 double-byte characters (6879 characters).

This module provides some functions to map from Shift_JIS to Unicode, and vice versa.

Conversion from Shift_JIS to Unicode

If the first parameter is a reference, that is used for coping with Shift_JIS characters unmapped to Unicode, SJIS_CALLBACK. (any reference will not allowed as STRING.)

If SJIS_CALLBACK is given, STRING is the second parameter; otherwise the first.

If SJIS_CALLBACK is not specified, Shift_JIS characters unmapped to Unicode are silently deleted and illegal bytes are skipped by one byte. (as if a coderef constantly returning null string, sub {''}, is passed as SJIS_CALLBACK.)

Currently, only coderefs are allowed as SJIS_CALLBACK. A string returned from SJIS_CALLBACK is inserted in place of the unmapped character or the illegal byte.

A coderef as SJIS_CALLBACK is called with one or more arguments.

If illegal byte appears (i.e. a leading byte [0x81..0x9F, 0xE0..0xEF] without trailing byte ([0x40..0x7E, 0x80..0xFC]), or a reserved byte ([0x80, 0xA0, 0xF0..0xFF]), the first argument is undef and the second argument is an unsigned integer representing the byte.

If an unmapped character appears, the first argument is a defined string representing a character.

Example

    my $sjis_callback = sub {
        my ($char, $byte) = @_;
        return function($char) if defined $char;
        die sprintf "illegal byte 0x%02x", $byte;
    };

In the example above, $char may be one of "\x82\xf2", "\xef\xfc".

The return value of SJIS_CALLBACK must be legal in the target format. E.g. never use with shiftjis_to_utf16be() a callback that returns UTF-8. I.e. you should prepare SJIS_CALLBACK for each UTF.

shiftjis_to_utf8([SJIS_CALLBACK,] STRING): Converts Shift_JIS to UTF-8.
shiftjis_to_unicode([SJIS_CALLBACK,] STRING): Converts Shift_JIS to Unicode. (Perl's internal format, flagged with SVf_UTF8, see perlunicode)
shiftjis_to_utf16le([SJIS_CALLBACK,] STRING): Converts Shift_JIS to UTF-16LE.
shiftjis_to_utf16be([SJIS_CALLBACK,] STRING): Converts Shift_JIS to UTF-16BE.
shiftjis_to_utf32le([SJIS_CALLBACK,] STRING): Converts Shift_JIS to UTF-32LE.
shiftjis_to_utf32be([SJIS_CALLBACK,] STRING): Converts Shift_JIS to UTF-32BE.

Conversion from Unicode to Shift_JIS

If the first parameter is a reference, that is used for coping with Unicode characters unmapped to Shift_JIS, UNICODE_CALLBACK. (any reference will not allowed as STRING.)

If UNICODE_CALLBACK is given, STRING is the second parameter; otherwise the first.

If UNICODE_CALLBACK is not specified, Shift_JIS characters unmapped to Unicode are silently deleted and partial bytes are skipped by one byte. (as if a coderef constantly returning null string, sub {''} is passed as UNICODE_CALLBACK.)

Currently, only coderefs are allowed as UNICODE_CALLBACK. A string returned from the coderef is inserted in place of the unmapped character.

A coderef as UNICODE_CALLBACK is called with one or more arguments. If the unmapped character is a partial character (an illegal byte), the first argument is undef and the second argument is an unsigned integer representing the byte. If not partial, the first argument is an unsigned interger representing a Unicode code point.

For example, characters unmapped to Shift_JIS are converted to numerical character references for HTML 4.01.

    sub toHexNCR {
        my ($char, $byte) = @_;
        return sprintf("&#x%x;", $char) if defined $char;
        die sprintf "illegal byte 0x%02x", $byte;
    }

    $sjis_string = utf8_to_shiftjis   (\&toHexNCR, $utf8_string);
    $sjis_string = unicode_to_shiftjis(\&toHexNCR, $unicode_string);
    $sjis_string = utf16le_to_shiftjis(\&toHexNCR, $utf16le_string);

The return value of UNICODE_CALLBACK must be legal in Shift_JIS.

utf8_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-8 to Shift_JIS.
unicode_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts Unicode to Shift_JIS.; This Unicode is coded in the Perl's internal format (see perlunicode). If SVf_UTF8 is not turned on, STRING is upgraded as an ISO 8859-1 (latin1) string.
utf16_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-16 (with or w/o BOM) to Shift_JIS.
utf16le_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-16LE to Shift_JIS.
utf16be_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-16BE to Shift_JIS.
utf32_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-32 (with or w/o BOM) to Shift_JIS.
utf32le_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-32LE to Shift_JIS.
utf32be_to_shiftjis([UNICODE_CALLBACK,] STRING): Converts UTF-32BE to Shift_JIS.

Export

By default:

    shiftjis_to_utf8     utf8_to_shiftjis
    shiftjis_to_utf16le  utf16le_to_shiftjis
    shiftjis_to_utf16be  utf16be_to_shiftjis
    shiftjis_to_unicode  unicode_to_shiftjis

On request:

    shiftjis_to_utf32le  utf32le_to_shiftjis
    shiftjis_to_utf32be  utf32be_to_shiftjis
                         utf16_to_shiftjis [*]
                         utf32_to_shiftjis [*]

[*] Their counterparts shiftjis_to_utf16() and shiftjis_to_utf32() are not implemented yet. They need more investigation on return values from SJIS_CALLBACK... (concatenation needs recognition of and coping with BOM)

AUTHOR

SADAHIRO Tomoyuki <SADAHIRO@cpan.org>

  Copyright(C) 2002-2007, SADAHIRO Tomoyuki. Japan. All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.