Logging and BOM

[mike-burns]

I have this LogConfig.yml:

loglevel: debug
filepath: log/authorize_net.log
maskSensitiveData: false

I'm seeing this error:

log writing failed. "\xEF" from ASCII-8BIT to UTF-8

Expanding it further:

[2020-01-29 14:04:30] DEBUG (sdk-ruby-2.0.0): \xEF\xBB\xBF<?xml version=\"1.0\" encoding=\"utf-8\"?>...

That's a UTF-8 BOM that it's trying to log.

---

Interestingly, I'm only seeing that error when I run my smoketest. When I use my Rails app, it logs the BOM to the file directly without complaint. That is to say, the log file has BOM characters throughout, but for some reason only when logging from a running Rails app.

Either way, it shouldn't have BOM characters scattered throughout the file.

---

One possible fix is to open the log file as 8-bit ASCII. For example, in the weirdly-named LogHelper.rb:

          filepath = File.open(cnf['filepath'], "a:ASCII-8BIT")
          @logger = Logger.new(filepath)

I tried to trace through why Ruby is making any of the decisions it made around this string (opening the file as UTF-8, converting from ASCII to UTF-8, etc.) but got lost around io.c's make_writeconv.

Perhaps the better solution is to convert respXml.body to UTF-8 sooner, just before logging it, in api_transaction.rb:

          LogHelper.log.debug(
            respXml.body.
              gsub("\xEF\xBB\xBF".force_encoding("ASCII-8BIT"), '').
              encode!(Encoding::UTF_8)
          )