Skip to content

[Improvement-17788] [TaskPlugin] Optimization of log processing for RemoteShellTask#17800

Merged
SbloodyS merged 16 commits intoapache:devfrom
qiong-zhou:Improvement-17788
Jan 12, 2026
Merged

[Improvement-17788] [TaskPlugin] Optimization of log processing for RemoteShellTask#17800
SbloodyS merged 16 commits intoapache:devfrom
qiong-zhou:Improvement-17788

Conversation

@qiong-zhou
Copy link
Copy Markdown
Contributor

@qiong-zhou qiong-zhou commented Dec 16, 2025 •
edited by SbloodyS
Loading

Purpose of the pull request

Optimization of log processing for RemoteShellTask. Reading the logs by line to avoid large string. close #17788

Brief change log

Modified RemoteExecutor.java

Verify this pull request

This change added tests and can be verified as follows:

  • Added testTrack() in RemoteExecutorTest to verify the change.

@qiong-zhou
Copy link
Copy Markdown
Contributor Author

qiong-zhou commented Dec 17, 2025

@SbloodyS WorkflowInstanceFailoverTestCase.testGlobalFailover_runningWorkflow_fromAnotherMaster failed.
This unit test failure appears to be unrelated to my changes. Please help me out.

@SbloodyS
Copy link
Copy Markdown
Member

SbloodyS commented Dec 17, 2025 •
edited
Loading

@SbloodyS WorkflowInstanceFailoverTestCase.testGlobalFailover_runningWorkflow_fromAnotherMaster failed. This unit test failure appears to be unrelated to my changes. Please help me out.

I've already rerun all tests.

@SbloodyS SbloodyS added this to the 3.4.0 milestone Dec 17, 2025
@SbloodyS SbloodyS added the improvement make more easy to user or prompt friendly label Dec 17, 2025
@qiong-zhou
Copy link
Copy Markdown
Contributor Author

qiong-zhou commented Dec 17, 2025

@SbloodyS WorkflowInstanceFailoverTestCase.testGlobalFailover_runningWorkflow_fromAnotherMaster failed. This unit test failure appears to be unrelated to my changes. Please help me out.

I've already rerun all tests.

@SbloodyS Unit tests failed again, any idea what might be causing this?

@ruanwenjun
Copy link
Copy Markdown
Member

ruanwenjun commented Dec 18, 2025

@qiong-zhou CI issue fixed by #17807

@qiong-zhou
Copy link
Copy Markdown
Contributor Author

qiong-zhou commented Dec 19, 2025

@ruanwenjun ok thanks.
@SbloodyS I've merged #17807 Please rerun the checks.
I'm wandering if I can rerun checks by myself?

@ruanwenjun
Copy link
Copy Markdown
Member

ruanwenjun commented Dec 22, 2025

@ruanwenjun ok thanks. @SbloodyS I've merged #17807 Please rerun the checks. I'm wandering if I can rerun checks by myself?

Right now, the CI need committer approve, this is primarily to reduce unnecessary CI resource consumption. If we can move the CI action to developer's repo, then we can remove this.

@SbloodyS SbloodyS modified the milestones: 3.4.0, 3.4.1 Dec 23, 2025
@qiong-zhou
Copy link
Copy Markdown
Contributor Author

qiong-zhou commented Dec 29, 2025 •
edited
Loading

@SbloodyS Please review the code.

SbloodyS
SbloodyS approved these changes Jan 9, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jan 12, 2026

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@SbloodyS SbloodyS merged commit 5c5b44c into apache:dev Jan 12, 2026
75 checks passed
@qiong-zhou qiong-zhou deleted the Improvement-17788 branch January 21, 2026 09:04
This was referenced May 1, 2026
Closed
Merged
ruanwenjun pushed a commit to leocook/dolphinscheduler that referenced this pull request May 2, 2026
@leocook @ruanwenjun
[Fix-18201][TaskPlugin] Fix RemoteShell task NullPointerException and…
b314cde 
… Pipe closed error

Revert the channel output reading approach introduced in apache#17800 from
getInvertedOut() back to setOut(ByteArrayOutputStream), which fixes two
regression bugs:
1. NPE: getInvertedOut() returns null before channel.open() completes
2. Pipe closed after 0 cycles: ChannelPipedInputStream throws IOException
   when command produces no output
leocook added a commit to leocook/dolphinscheduler that referenced this pull request May 3, 2026
@leocook
[Fix-18201][TaskPlugin] Fix RemoteShell task NullPointerException and…
7554726 
… Pipe closed error

Revert the channel output reading approach introduced in apache#17800 from
getInvertedOut() back to setOut(ByteArrayOutputStream), which fixes two
regression bugs:
1. NPE: getInvertedOut() returns null before channel.open() completes
2. Pipe closed after 0 cycles: ChannelPipedInputStream throws IOException
   when command produces no output
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhongjiajie zhongjiajie zhongjiajie approved these changes

@SbloodyS SbloodyS SbloodyS approved these changes

@caishunfeng caishunfeng Awaiting requested review from caishunfeng caishunfeng is a code owner

@Gallardot Gallardot Awaiting requested review from Gallardot Gallardot is a code owner

@ruanwenjun ruanwenjun Awaiting requested review from ruanwenjun

Assignees

@qiong-zhou qiong-zhou

Labels

backend improvement make more easy to user or prompt friendly ready-to-merge test

Projects

None yet

Milestone

3.4.1

Development

Successfully merging this pull request may close these issues.

[Improvement][Task] Optimization of log processing for RemoteShellTask

4 participants

@qiong-zhou @SbloodyS @ruanwenjun @zhongjiajie