Skip to content

Transformer to convert characters that cannot be converted to another string when converting character codes with golang's transform.Transformer.

License

Notifications You must be signed in to change notification settings

tomtwinkle/garbledreplacer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

garbledreplacer

GitHub Go Report Card GitHub Workflow Status GitHub Workflow Status

Overview

transform.Transformer to replace characters that cannot be converted from UTF-8 to another character encoding.

UTF-8 から別の文字コードに変換する際、変換出来ない文字を別の文字に置き換えるための transform.Transformer

Motivation

Golang標準の japanese.ShiftJIS 等のEncoderでは変換出来ない文字が合った場合 rune not supported by encoding errorが出てしまい変換ができない。

そのため、Encoderを通す前に変換できない文字を事前に別の文字に置き換える為のTransformerを作成した。

japanese.ShiftJIS japanese.EUCJP traditionalchinese.Big5 などのEncoderの前にtransform.Chainで動作する薄いwrapperとなっている。

Usage

const msg = "一二三四🍣五六七八九🍺十拾壱"

var buf bytes.Buffer
w := transform.NewWriter(&buf, garbledreplacer.NewTransformer(japanese.ShiftJIS, '?'))
if _, err := w.Write([]byte([]byte(msg))); err != nil {
    panic(err)
}
if err := w.Close(); err != nil {
    panic(err)
}

fmt.Println(buf.String())
// 一二三四?五六七八九?十拾壱

About

Transformer to convert characters that cannot be converted to another string when converting character codes with golang's transform.Transformer.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages