Commons Lang 3 -- Improved and Powerful StringEscapeUtils

In the first and second parts of this series I talked about some of the new features like enum and concurrency support that have been added in commons-lang 3. In this article, I am going to talk about a new package 'org.apache.commons.lang3.text.translate' which has been added in commons-lang 3. This package is added to fix problems in the design and implementation of the StringEscapeUtils class which exists in versions prior to 3.0. To make it clearer, let's first talk about the purpose of StringEscapeUtils class and the problems it had prior to version 3.

Purpose of StringEscapeUtils

StringEscapeUtils is a utility class which escapes and unescapes String for Java, JavaScript, HTML, XML, and SQL. For example,

      @Test
	public void test_StringEscapeUtils() {
		assertEquals("\\\\\\n\\t\\r", StringEscapeUtils.escapeJava("\\\n\t\r")); // escapes the Java String
		assertEquals("\\\n\t\r",StringEscapeUtils.unescapeJava("\\\\\\n\\t\\r")); //unescapes the Java String
		assertEquals("I didn\\'t say \\\"you to run!\\\"",StringEscapeUtils.escapeJavaScript("I didn't say \"you to run!\""));//escapes the Javascript
		assertEquals("&lt;xml&gt;", StringEscapeUtils.escapeXml("<xml>"));//escapes the xml
	}

 Problems with StringEscapeUtils

There were a lot of problems in the StringEscapeUtils implementation prior to version3. Some of these were:

• The implementation was not extensible. Let's take an example of escapeJava, suppose we want to add support in the escapeJava method that it should start escaping single quotes. To add such support we would have to change the existing class code and another if condition which if satisfied will escape single quotes. So, the API was breaking the open-closed principle i.e. a class should be open for extension and closed for modification.
• It was not symmetric i.e. original should be equal to unescape(escape(original))  but it was not the case.
• StringEscapeUtils.escapeHtml() escapes multibyte characters like Chinese, Japanese etc. Issue 339

  @Test
  	public void testEscapeHiragana() {
  		// Some random Japanese unicode characters
  		String original = "\u304B\u304C\u3068";
  		String escaped = StringEscapeUtils.escapeHtml(original);
  		assertEquals(original, escaped);
  	}

• StringEscapeUtils.escapeHtml incorrectly converts unicode characters above U+00FFFF into 2 characters. Issue 480

  @Test
  	public void testEscapeHtmlHighUnicode()	throws java.io.UnsupportedEncodingException {
  		byte[] data = new byte[] { (byte) 0xF0, (byte) 0x9D, (byte) 0x8D,(byte) 0xA2 };
  		String original = new String(data, "UTF8");
  		String escaped = StringEscapeUtils.escapeHtml(original);
  		assertEquals(original, escaped);
  	}

• StringEscaper.escapeXml() escapes characters > 0x7f . Issue 66

  @Test
  	public void shouldNotEscapeValuesGreaterThan0x7f() {
  		assertEquals("XML should not escape >0x7f values", "\u00A1",StringEscapeUtils.escapeXml("\u00A1"));
  	}

public static final CharSequenceTranslator ESCAPE_JAVA = new AggregateTranslator(new LookupTranslator(
            new String[][] {
              {"\"", "\\\""},
              {"\\", "\\\\"},
          }),new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE()),UnicodeEscaper.outsideOf(32, 0x7f));

 and in the escapeJava method

public static final String escapeJava(String input) {
        return ESCAPE_JAVA.translate(input);
    }

 A constant of type CharSequenceTranslator was assigned an AggregateTranslator object. AggregateTranslator can take an array of translators, and it iterates over each of them. The LookupTranslator replaces the string at zeroth index with the string at the first index. UnicodeEscaper translates values outside the given range to unicode values. As you can see, you can very easily write your own escape methods.  For example, if you want to add the support of escaping &, you can do it like this

public static final CharSequenceTranslator ESCAPE_JAVA =
          new LookupTranslator(
            new String[][] {
              {"\"", "\\\""},
              {"\\", "\\\\"},
          }).with(new LookupTranslator(
        		  new String[][]{
        		{"&", "&"},
        		{"<", "<"}}
          )).with(
            new LookupTranslator(EntityArrays.JAVA_CTRL_CHARS_ESCAPE())
          ).with(
            UnicodeEscaper.outsideOf(32, 0x7f)
        );