Skip to content

MySql support.#15

Merged
PhenX merged 19 commits intoPhenX:masterfrom
SebastianStehle:mysql
May 21, 2025
Merged

MySql support.#15
PhenX merged 19 commits intoPhenX:masterfrom
SebastianStehle:mysql

Conversation

@SebastianStehle
Copy link
Copy Markdown
Collaborator

@SebastianStehle SebastianStehle commented May 21, 2025

Closes #6

@SebastianStehle SebastianStehle requested a review from PhenX as a code owner May 21, 2025 07:09
@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

MySQL support.

I would like to reuse the test containers because it is really slow to restart them all the time. But the tests are not fully ready yet, but as preparation I created more random test data.

@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

Is it possible to add a benchmark too ?

…workCore.BulkInsert into mysql

# Conflicts:
#	src/PhenX.EntityFrameworkCore.BulkInsert/Options/BulkInsertOptions.cs
@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

That's embarassing, it looks like the pipeline does not detect any test so it always tells that it's OK while it's not, with this pull request

@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

Benchmark added :)

@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

I hope I have fixed all tests.

There are few things worth to mention:

  1. MySQL does not allow explicit conflicts, only general conflict statements. Therefore I tried to get a scenario running where no Match was defined. This is actually what I also need for upsert statements. Unfortunately SQL Server and Postgres do not allow it without a match, so I guess we have to find the primary keys for that.
  2. MySQL has no support for "RETURNING" and no other methods to return the statements.
  3. Autoincremented values are not added by default to the temp table, therefore you can also not find conflicts on that. I have added an option for that.
  4. The tests were painfully slow and I could not run 10 sql servers with docker without docker causign problems. Therefore I have created a fixture for that.

@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

Mhm, I have this locking issues, that I can not reproduce locally. Would be great if you could also have a look.

@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

Sure! I'll take a look asap

@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

@SebastianStehle that's OK now : test are not ran in parallel anymore but containers are started only once per test class per target framework

@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

I actually prefer the current solution with fixtures because it is super fast to run locally, but I think it is not related to the actual problem with the locking.

@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

They are now actually IClassFixtures, it's still super fast to run locally, it seems to solve the locking issue two : too many containers where started at the same time

…workCore.BulkInsert into mysql

# Conflicts:
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContainer/TestDbContainer.cs
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/DbContainer/TestDbContainerSqlServer.cs
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Basic/BasicTestsBase.cs
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Basic/BasicTestsPostgreSql.cs
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Basic/BasicTestsSqlServer.cs
#	tests/PhenX.EntityFrameworkCore.BulkInsert.Tests/Tests/Basic/BasicTestsSqlite.cs
@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

The locking issue only happened with mysql and I think the problem was that the transaction was not disposed. At least it is working fine now. Everything is green and waiting for review.

Comment thread src/PhenX.EntityFrameworkCore.BulkInsert.MySql/MySqlBulkInsertProvider.cs Outdated
Comment thread src/PhenX.EntityFrameworkCore.BulkInsert.MySql/MySqlDbContextOptionsExtensions.cs Outdated
Comment thread src/PhenX.EntityFrameworkCore.BulkInsert.SqlServer/SqlServerBulkInsertProvider.cs Outdated
protected override IDatabaseContainer? GetDbContainer()
{
return new MySqlBuilder()
.WithReuse(true)
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure reuse works now, because of .WithName(GetRandomContainerName())

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, does not work fine together. I think the reuse is really worth it as it is even faster. I have a large test suite similar to with way more tests and it was such a good improvement.

}

public async Task DisposeAsync()
public async Task<TDbContext> CreateContextAsync()
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all your changes required for mysql provider to work ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, I added a few more tests, but most of the stuff is needed because of the "reuse" to isolate the tests logically from each other. It could be also solved by creating a new database with each run, but I like this, because it is easier to debug. You can just restart the container and connect with pgadmin or other tools.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok fair enough

@PhenX PhenX merged commit bfa007e into PhenX:master May 21, 2025
1 check passed
@SebastianStehle
Copy link
Copy Markdown
Collaborator Author

Nice, thank you very much for merging the branch and for your good work :)

@PhenX
Copy link
Copy Markdown
Owner

PhenX commented May 21, 2025

@SebastianStehle and thank you for your contribution :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MySQL support

2 participants