Skip to content

JSPWIKI-1295 see jira for details #522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
spyhunter99 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

JSPWIKI-1295 see jira for details #522

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions ...iki-tika-searchprovider/src/main/java/org/apache/wiki/search/tika/TikaSearchProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Licensed to the Apache Software Foundation (ASF) under one
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.tika.exception.TikaException;
import org.apache.tika.exception.WriteLimitReachedException;
import org.apache.tika.metadata.ClimateForcast;
import org.apache.tika.metadata.CreativeCommons;
import org.apache.tika.metadata.Database;
Expand Down Expand Up @@ -57,6 +58,13 @@ public class TikaSearchProvider extends LuceneSearchProvider {
final AutoDetectParser parser;
final Set< String > textualMetadataFields;

/**
* Maximum number of characters extracted from a single attachment for indexing. Bounds the memory used at index
* time regardless of attachment content: a small compressed attachment (e.g. a zip bomb, which expands to
* gigabytes of text) must not be able to grow into an unbounded in-memory string and exhaust the heap.
*/
static final int MAX_EXTRACTED_CHARS = 1_000_000;

public TikaSearchProvider() {
parser = new AutoDetectParser();

Expand Down Expand Up @@ -99,10 +107,17 @@ protected String getAttachmentContent( final Attachment att ) {
final Metadata metadata = new Metadata();
metadata.set( TikaCoreProperties.RESOURCE_NAME_KEY, att.getFileName() );

final ContentHandler handler = new BodyContentHandler(-1 );
// -1 disables the character size limit; otherwise only the first 100.000 characters are indexed

parser.parse( attStream, handler, metadata );
final ContentHandler handler = new BodyContentHandler( MAX_EXTRACTED_CHARS );
try {
parser.parse( attStream, handler, metadata );
} catch( final SAXException e ) {
if( !WriteLimitReachedException.isWriteLimitReached( e ) ) {
throw e;
}
// keep what was extracted so far, so oversized attachments still get (truncated) indexing
LOG.warn( "Attachment {} reached the maximum of {} extracted characters, indexing truncated content",
att.getFileName(), MAX_EXTRACTED_CHARS );
}
out.append( handler );

final String[] names = metadata.names();
Expand Down
Loading