uchmviewer doesn't display Japanese .chm files correctly

[perdrix52]

Japanese chm files are typically encoded as Shift-JIS. kchmviewer has a View/Encoding menu option that allows me to specify the file encoding so that it displays correctly.

uchmviewer doesn't seem to allow me to do that so my Japanese help which should look like:

is instead displayed as:

which isn't too great.

Can this be addressed please.

[u-235]

I'm not entirely sure what needs to be fixed. I just checked, and the encoding choose menu is still available for CHM files.

[perdrix52]

Silly me - this was macOS so the Menu stuff isn't part of the application but is shown at the top of the screen.

However is there any chance the if the language is set to Japanese then the default encoding for chm files should be set to Shift-JIS

[gonwan]

@perdrix52

However is there any chance the if the language is set to Japanese then the default encoding for chm files should be set to Shift-JIS

This is already the case. uchmviewer does detect and set encodings accordingly. But the encoding here sets both the internal encoding used to decode/display chm menu and index, and the encoding used to display a single html page. If these two encodings do not agree, garbage text is shown.

Maybe it is just fine to let the embeded browser detect the page encoding automatically, as in most modern browsers in 2026.

[u-235]

Maybe it is just fine to let the embeded browser detect the page encoding automatically, as in most modern browsers in 2026.

That's an interesting idea, but it only works if the page includes a valid META tag specifying the character encoding. Unfortunately, some pages lack this information.

[gonwan]

@u-235
That is not quite true. Chrome is able to correctly display following two html pages, even without help of meta tag or http headers.

<!-- save in gbk encoding -->
zhcn_gbk<br/>
123abc<br/>
中文

<!-- save in shiftjis encoding -->
ja_shiftjis<br/>
123abc<br/>
日本語てす

However, a character detector is not a standard part of chrome/firefox/qtwebkit/qtwebengine. They use different aux libraries. See:

• https://superuser.com/questions/1160003/how-do-i-change-the-character-encoding-for-a-webpage-in-chrome
• https://stackoverflow.com/questions/43148464/how-do-browsers-determine-the-encoding-used

[gonwan]

@u-235
I created a chm file for verify. First apply the smallest patch to disable manual encoding settings:

diff --git a/src/qtwebengine/dataprovider.cpp b/src/qtwebengine/dataprovider.cpp
index def2973..4714885 100644
--- a/src/qtwebengine/dataprovider.cpp
+++ b/src/qtwebengine/dataprovider.cpp
@@ -93,8 +93,8 @@ void DataProvider::requestStarted( QWebEngineUrlRequestJob* request )
 	// This is the easiest and most stable way to set the encoding.
 	if ( mimetype == "text/html" )
 	{
-		buf.prepend( QString( "<META http-equiv='Content-Type' content='text/html; charset=%1'>" )
-		             .arg( ::mainWindow->chmFile()->currentEncoding() ).toLatin1() );
+//		buf.prepend( QString( "<META http-equiv='Content-Type' content='text/html; charset=%1'>" )
+//		             .arg( ::mainWindow->chmFile()->currentEncoding() ).toLatin1() );
 	}
 
 	// We will use the buffer because reply() requires the QIODevice.
diff --git a/src/qtwebkit/dataprovider.cpp b/src/qtwebkit/dataprovider.cpp
index ca3d01e..46ac6f1 100644
--- a/src/qtwebkit/dataprovider.cpp
+++ b/src/qtwebkit/dataprovider.cpp
@@ -93,9 +93,7 @@ QByteArray KCHMNetworkReply::loadResource( const QUrl& url )
 
 	if ( mime == "text/html" || mime == "text/xhtml" || mime == "text/xml" )
 	{
-		QString header = QString( "%1; charset=%2" )
-		                 .arg( mime )
-		                 .arg( ::mainWindow->chmFile()->currentEncoding() );
+		QString header = mime;
 		setHeader( QNetworkRequest::ContentTypeHeader, header );
 	}
 

The attched chm file has mixed encodings in separated pages. UTF-8/GBK/Shift-JIS all display correctly. And the QtWebengine version even works without the meta tag. It does make things better. I'll make a PR to address the issue if you agree with me. In addition, The view source function should also be fixed.

testautodetect.chm.zip

[u-235]

I'm not ready to make changes related to page encoding just yet. I've almost finished converting the HTML rendering engines used by the app into plugins. And in the current version, these plugins only accept HTML in Unicode, which means the app is responsible for the encoding conversion.

[perdrix52]

@u-235 I'm creating my Japanese help files in utf-8 and there's a meta tag to say so. They display perfectly with Windows HtmlHelp, but not so much with uchmviewer. I was generating them in Shift_JIS but had problems with invalid characters in the help display (right panel). Specifically '〜' which displayed as '?', hence the change to UTF-8. With uchmviewer set to Shift-JIS, then the navigator panel displays correctly, but the actual help files are wrong. If I use View/Set Encoding to UTF8, then the navigator panel displays incorrectly (I think that has to be in Shift-JIS), but the help contents are shown correctly (but only after I reselect the relevant section in the navigator).

This is a bit of a rock and a hard-place problem. I can either get the navigator panel displayed correctly, or the actual help itself, but not both :(

Hope you can offer a solution in the short term.

kchmviewer doesn't seem to have this problem - displays everything just fine on Linux, but I use uchmviewer on macOS.