utf-8 word boundary regex in javascript

Posted on

utf-8 word boundary regex in javascript – Even if we have a good project plan and a logical concept, we will spend the majority of our time correcting errors abaout javascript and regex. Furthermore, our application can run without obvious errors with JavaScript, we must use various ways to ensure that everything is operating properly. In general, there are two types of errors that you’ll encounter while doing something wrong in code: Syntax Errors and Logic Errors. To make bug fixing easier, every JavaScript error is captured with a full stack trace and the specific line of source code marked. To assist you in resolving the JavaScript error, look at the discuss below to fix problem about utf-8 word boundary regex in javascript.

Problem :

In JavaScript:

"ab abc cab ab ab".replace(/babb/g, "AB");

correctly gives me:

"AB abc cab AB AB"

When I use utf-8 characters though:

"αβ αβγ γαβ αβ αβ".replace(/bαβb/g, "AB");

the word boundary operator doesn’t seem to work:

"αβ αβγ γαβ αβ αβ"

Is there a solution to this?

Solution :

The word boundary assertion does only match if a word character is not preceded or followed by another word character (so .b. is equal to Ww and wW). And w is defined as [A-Za-z0-9_]. So w doesn’t match greek characters. And thus you cannot use b for this case.

What you could do instead is to use this:

"αβ αβγ γαβ αβ αβ".replace(/(^|s)αβ(?=s|$)/g, "$1AB")

Not all Javascript regexp implementation has support for Unicode ad so you need to escape it

"αβ αβγ γαβ αβ αβ".replace(/u03b1u03b2/g, "AB"); // "AB ABγ γAB AB AB"

For mapping the characters you can take a look at http://htmlhelp.com/reference/html40/entities/symbols.html

Of course, this doesn’t help with the word boundary issue (as explained in other answers) but should at least enable you to match the characters properly

I needed something to be programmable and handle punctuation, brackets, etc.

http://jsfiddle.net/AQvyd/

var wordToReplace = '買い手',
    replacementWord = '[[BUYER]]',
    text = 'Mange 買い手 information. The selected Store and Classification will be the default on the สั่งซื้อ.'

function replaceWord(text, wordToReplace, replacementWord) {
    var re = new RegExp('(^|\s|\(|'|"|,|;)' + wordToReplace + '($|\s|\)|\.|'|"|!|,|;|\?)', 'gi');
    return text.replace(re, replacementWord);
}

I’ve written a javascript resource editor so this is why I’ve found this page and also answered it out of necessity since I couldn’t find a word boundary parametarized regexp that worked well for Unicode.

Not all the implementations of RegEx associated with Javascript engines a unicode aware.

For example Microsofts JScript using in IE is limited to ANSI.

When you’re dealing with Unicode and natural-language words, you probably want to be more careful with boundaries than just using b. See this answer for details and directions.

Leave a Reply

Your email address will not be published. Required fields are marked *