Skip to content

feat: Adds 'skip-empty-xacts' support#248

Closed
adrijshikhar wants to merge 4 commits into
eulerto:masterfrom
adrijshikhar:master
Closed

feat: Adds 'skip-empty-xacts' support#248
adrijshikhar wants to merge 4 commits into
eulerto:masterfrom
adrijshikhar:master

Conversation

@adrijshikhar

Copy link
Copy Markdown

fixes #106

@adrijshikhar

Copy link
Copy Markdown
Author

@eulerto Please have a look and let me know what changes are required

@eulerto eulerto left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial review.

Comment thread README.md
Comment thread sql/skip_empty_xacts.sql Outdated
Comment thread sql/skip_empty_xacts.sql Outdated
Comment thread sql/skip_empty_xacts.sql
-- predictability
SET synchronous_commit = on;

SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'wal2json');

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these tests exercise the proposal? Why don't you use simple statements (for example, using filter-tables or add-tables option)? You should include tests for both formats (see other test files).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Comment thread wal2json.c Outdated
Comment thread wal2json.c Outdated

/* Transaction starts */
OutputPluginPrepareWrite(ctx, true);
OutputPluginPrepareWrite(ctx, last_write);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test this modification with the write-in-chunks option?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no i have not
i will benchmark this change with write-in-chunks option

Comment thread wal2json.c Outdated
Comment thread wal2json.c Outdated
Comment thread wal2json.c Outdated
/* version 2 */
static void pg_decode_begin_txn_v2(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn);
ReorderBufferTXN *txn, bool last_write);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an additional space here.

Comment thread wal2json.c
* has requested to skip the empty transactions we can skip the empty streams
* even though the transaction has written some changes.
*/
typedef struct

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need this new struct? Can't you use data->nr_changes?

@adrijshikhar

Copy link
Copy Markdown
Author

@eulerto can you please go through the comments I posted?
Thanks

@eulerto

eulerto commented Sep 20, 2022

Copy link
Copy Markdown
Owner

I don't follow. You asked a question about pgindent. AFAICS you didn't post another update solving all concerns. I suggest that you read Creating Clean Patches and Development Tools and Help (there is a subsection called What's the formatting style used in PostgreSQL source code?.

@adrijshikhar

adrijshikhar commented Sep 22, 2022

Copy link
Copy Markdown
Author

thanks I will go through that. I have addressed all of your comments and made changes accordingly. I am stuck on pgindent. Somehow it changes all of the indentations. I will try to fix it and then update you.

@eulerto

eulerto commented Sep 22, 2022

Copy link
Copy Markdown
Owner

If it is changing a lot, don't run it. Instead, fix some superfluous changes (such as whitespace and new line additions). I can clean it up later, if required.

@adrijshikhar

Copy link
Copy Markdown
Author

yeah sure thanks

@adrijshikhar adrijshikhar closed this by deleting the head repository Jun 2, 2026
@adrijshikhar

Copy link
Copy Markdown
Author

This PR can't be reopened (GitHub reports the submitting fork was deleted). I reworked the feature from scratch on top of current master, addressed all the 2022 review feedback, and opened #293 to supersede this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Getting an overwhelming amount of transactions with empty change sets despite using add-tables

2 participants