Skip to content

Fix generated-ID row mismatches in PrepAndExpectedTestCase #907

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jeffjensen merged 1 commit into from
Aug 4, 2026
Merged

Fix generated-ID row mismatches in PrepAndExpectedTestCase #907

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion src/changes/changes.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
</properties>

<body>
<release version="3.4.1-SNAPSHOT" date="TBD" description="A documentation-site overhaul (new tutorials, 9 database vendor guides, a class-by-class Core Components reference, consolidated Filters/Datasets/Operations pages, and a new Developing DbUnit contributor section covering coding standards, commit/changelog requirements, and the GitHub workflow); an opt-in all-column sort for tables without a primary key; MultiSchemaMySqlMetadataHandler for multi-schema MySQL connections; and multiple bug fixes including escape-pattern handling, primary-key filter fallback, empty-dataset DTD export, DatabaseDataSet initialization order, and a from-scratch clean-build pass across Javadoc doclint, Checkstyle, and compiler warnings">
<release version="3.4.1-SNAPSHOT" date="TBD" description="A documentation-site overhaul (new tutorials, 9 database vendor guides, a class-by-class Core Components reference, consolidated Filters/Datasets/Operations pages, and a new Developing DbUnit contributor section covering coding standards, commit/changelog requirements, and the GitHub workflow); an opt-in all-column sort for tables without a primary key; an opt-in sort-on-filtered-columns-only mode for DefaultPrepAndExpectedTestCase fixing false failures on tables with a generated/identity first column; MultiSchemaMySqlMetadataHandler for multi-schema MySQL connections; and multiple bug fixes including escape-pattern handling, primary-key filter fallback, empty-dataset DTD export, DatabaseDataSet initialization order, and a from-scratch clean-build pass across Javadoc doclint, Checkstyle, and compiler warnings">
<action dev="jeffjensen" type="add" issue="840" system="github" due-to="jeffjensen">
Add repo-root README.adoc, rendered natively by GitHub via Asciidoctor, so the repository landing page shows a pitch, build/reproducible-build badges, a pointer to the "dbUnit in 5 Minutes" tutorial, and links to the documentation site, Maven coordinates, GitHub Discussions, and CONTRIBUTING.md instead of nothing.
</action>
Expand Down Expand Up @@ -218,6 +218,9 @@
<action dev="jeffjensen" type="fix" issue="904" system="github" due-to="jeffjensen">
Fix the Javadoc @param/@return/@throws gaps across the main source tree that issue 902's from-scratch doclint pass left unaddressed since they are warnings, not build-failing errors: missing one-sentence summaries on classes/interfaces/methods/fields whose doc block was tag-only (doclint's "no main description"), entirely missing or description-less @param/@return/@throws tags, and missing field/method comments including on org.dbunit.assertion.DbComparisonFailure's and PropertyChangeMulticaster's private fields and readObject/writeObject methods, which doclint checks regardless of visibility once a class implements Serializable.
</action>
<action dev="jeffjensen" type="fix" issue="672" system="github" due-to="lufecir">
Fix DefaultPrepAndExpectedTestCase#verifyData false-failing tables whose generated/identity column is the first column and excluded from comparison: it always sorted the actual table by all of its native columns - identity column included - while the expected table, which never declares that column, sorted by data content only; excludeColumns/includeColumns were applied to the comparison but never to the sort. When production code does not guarantee row insertion order (e.g. Hibernate reordering a batch insert), the database's generated-ID assignment order then diverges from the data-content order, misaligning same-data rows and failing the comparison despite both sides holding identical data. Add VerifyTableDefinition#sortOnFilteredColumnsOnly (default false, preserving prior behavior), implementing the opt-in toggle proposed in issue 676: when true, both tables sort by only their excludeColumns/includeColumns-filtered columns instead of by all native columns.
</action>
</release>
<release version="3.4.0" date="Jul 28, 2026" description="Test-suite hardening (un-skip and strengthen dozens of disabled/no-op tests); add CachingConnectionProvider and reduce DefaultPrepAndExpectedTestCase's per-test connection churn; pin identifier case-folding to Locale.ENGLISH for Turkish-locale correctness; and a broad set of correctness fixes across export formats (XML, YAML, CSV, XLS, Ant), TimestampDataType timezone handling, InsertOperation/TransactionOperation, and resource-leak cleanups">
<action dev="jeffjensen" type="fix" issue="797" system="github" due-to="jeffjensen">
Expand Down
137 changes: 131 additions & 6 deletions src/main/java/org/dbunit/DefaultPrepAndExpectedTestCase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,13 +702,24 @@ protected void verifyData(final IDatabaseConnection connection,
verifyTableDefinition.getColumnValueComparers();
final ValueComparer defaultValueComparer =
verifyTableDefinition.getDefaultValueComparer();
final boolean sortOnFilteredColumnsOnly =
verifyTableDefinition.isSortOnFilteredColumnsOnly();

final ITable expectedTable = loadTableDataFromDataSet(tableName);
final ITable actualTable =
loadTableDataFromDatabase(tableName, connection);

verifyData(expectedTable, actualTable, excludeColumns, includeColumns,
defaultValueComparer, columnValueComparers);
if (sortOnFilteredColumnsOnly)
{
verifyData(expectedTable, actualTable, excludeColumns,
includeColumns, defaultValueComparer, columnValueComparers,
true);
} else
{
verifyData(expectedTable, actualTable, excludeColumns,
includeColumns, defaultValueComparer,
columnValueComparers);
}
}

/**
Expand Down Expand Up @@ -796,13 +807,62 @@ public ITable loadTableDataFromDatabase(final String tableName,
* {@link ValueComparer}. Can be <code>null</code> and will
* default to defaultValueComparer for all columns in all tables.
* @throws DatabaseUnitException if the tables' row counts, columns, or data do not match.
* @see #verifyData(ITable, ITable, String[], String[], ValueComparer, Map, boolean)
* to also control whether sorting considers only the filtered
* columns; this overload always sorts by all native columns.
*/
protected void verifyData(final ITable expectedTable,
final ITable actualTable, final String[] excludeColumns,
final String[] includeColumns,
final ValueComparer defaultValueComparer,
final Map<String, ValueComparer> columnValueComparers)
throws DatabaseUnitException
{
verifyData(expectedTable, actualTable, excludeColumns, includeColumns,
defaultValueComparer, columnValueComparers, false);
}

/**
* For the specified expected and actual tables (and excluding and including
* the specified columns), verify the actual data is as expected.
*
* @param expectedTable
* The expected table to compare the actual table to.
* @param actualTable
* The actual table to compare to the expected table.
* @param excludeColumns
* The column names to exclude from comparison. See
* {@link org.dbunit.dataset.filter.DefaultColumnFilter#excludeColumn(String)}
* .
* @param includeColumns
* The column names to only include in comparison. See
* {@link org.dbunit.dataset.filter.DefaultColumnFilter#includeColumn(String)}
* .
* @param defaultValueComparer
* {@link ValueComparer} to use with column value comparisons
* when the column name for the table is not in the
* columnValueComparers {@link Map}. Can be <code>null</code> and
* will default.
* @param columnValueComparers
* {@link Map} of {@link ValueComparer}s to use for specific
* columns. Key is column name, value is the
* {@link ValueComparer}. Can be <code>null</code> and will
* default to defaultValueComparer for all columns in all tables.
* @param sortOnFilteredColumnsOnly
* True to sort the expected and actual tables by only the
* columns that survive excludeColumns/includeColumns, instead
* of by all of the actual table's native columns; see
* {@link VerifyTableDefinition#isSortOnFilteredColumnsOnly()}.
* @throws DatabaseUnitException if the tables' row counts, columns, or data do not match.
* @since 3.4.1
*/
protected void verifyData(final ITable expectedTable,
final ITable actualTable, final String[] excludeColumns,
final String[] includeColumns,
final ValueComparer defaultValueComparer,
final Map<String, ValueComparer> columnValueComparers,
final boolean sortOnFilteredColumnsOnly)
throws DatabaseUnitException
{
final String methodName = "verifyData";

Expand All @@ -815,16 +875,31 @@ protected void verifyData(final ITable expectedTable,
final Column[] expectedTableColumns = makeExpectedTableColumns(
actualTableColumns, expectedTableMetaData);

log.debug("{}: Sorting expected table using all columns", methodName);
final Column[] actualSortColumns;
final Column[] expectedSortColumns;
if (sortOnFilteredColumnsOnly)
{
log.debug("{}: Sorting using only filtered columns", methodName);
final String tableName = actualTableMetaData.getTableName();
actualSortColumns = makeSortColumns(actualTableColumns,
excludeColumns, includeColumns, tableName);
expectedSortColumns = makeSortColumns(expectedTableColumns,
excludeColumns, includeColumns, tableName);
} else
{
log.debug("{}: Sorting using all columns", methodName);
actualSortColumns = actualTableColumns;
expectedSortColumns = expectedTableColumns;
}

final SortedTable expectedSortedTable =
new SortedTable(expectedTable, expectedTableColumns, true);
new SortedTable(expectedTable, expectedSortColumns, true);
expectedSortedTable.setUseComparable(true);
log.trace("{}: Sorted expected table={}", methodName,
expectedSortedTable);

log.debug("{}: Sorting actual table using all columns", methodName);
final SortedTable actualSortedTable =
new SortedTable(actualTable, actualTableColumns);
new SortedTable(actualTable, actualSortColumns);
actualSortedTable.setUseComparable(true);
log.trace("{}: Sorted actual table={}", methodName, actualSortedTable);

Expand Down Expand Up @@ -855,6 +930,56 @@ protected void verifyData(final ITable expectedTable,
columnValueComparers);
}

/**
* Reduces the given columns to those that survive the given exclude and
* include column filters, using the same matching semantics - including
* {@link DefaultColumnFilter}'s wildcard pattern support - as
* {@link #applyColumnFilters(ITable, String[], String[])}, so the sort
* key always matches the columns that end up compared.
*
* @param columns
* The columns to filter.
* @param excludeColumns
* The column names to exclude; null or empty to exclude none.
* @param includeColumns
* The column names to only include; null to include all.
* @param tableName
* The table name; passed only to
* {@link DefaultColumnFilter#accept(String, Column)} for its
* debug logging.
* @return The filtered columns, in columns' original order.
*/
private Column[] makeSortColumns(final Column[] columns,
final String[] excludeColumns, final String[] includeColumns,
final String tableName)
{
final DefaultColumnFilter columnFilter = new DefaultColumnFilter();
if (includeColumns != null)
{
for (final String includeColumn : includeColumns)
{
columnFilter.includeColumn(includeColumn);
}
}
if (excludeColumns != null)
{
for (final String excludeColumn : excludeColumns)
{
columnFilter.excludeColumn(excludeColumn);
}
}

final List<Column> sortColumns = new ArrayList<>();
for (final Column column : columns)
{
if (columnFilter.accept(tableName, column))
{
sortColumns.add(column);
}
}
return sortColumns.toArray(new Column[sortColumns.size()]);
}

/**
* If expected column definitions exist and are {@link DataType.UNKNOWN},
* make them from actual table column definitions.
Expand Down
Loading
Loading