[Fix-17370] Fix worker server start failed when using hdfs storage type

[SbloodyS]

Purpose of the pull request

close #17370
close #17488

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ruanwenjun]

LGTM

[Kinsonx]

After adding '-Dclassifier=shade' based on 3.3.1-release version, still got the 'java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder' when start worker-server. It seems that jar file 'dolphinscheduler-storage-hdfs-3.3.1-shade.jar' dose not hive classes related to htrace. Did I miss any configurations ?

[Mrhs121]

After adding '-Dclassifier=shade' based on 3.3.1-release version, still got the 'java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder' when start worker-server. It seems that jar file 'dolphinscheduler-storage-hdfs-3.3.1-shade.jar' dose not hive classes related to htrace. Did I miss any configurations ?

You haven't missed any configuration. The current workaround solution is to copy hbase-noop-htrace-4.1.1.jar to worker-server/libs.

[ruanwenjun]

@Kinsonx Is your plugin dir exit dolphinscheduler-datasource-hive-3.3.1-shade.jar? hbase-noop-htrace should be shaded in this jar

[Mrhs121]

@Kinsonx Is your plugin dir exit dolphinscheduler-datasource-hive-3.3.1-shade.jar? hbase-noop-htrace should be shaded in this jar

I've created FOLLOWUP PR #17524 to fix it.

[Kinsonx]

@Kinsonx Is your plugin dir exit dolphinscheduler-datasource-hive-3.3.1-shade.jar? hbase-noop-htrace should be shaded in this jar

No datasource-plugins, only storage-plugins which contains 'storage-hdfs'

[Kinsonx]

After adding '-Dclassifier=shade' based on 3.3.1-release version, still got the 'java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder' when start worker-server. It seems that jar file 'dolphinscheduler-storage-hdfs-3.3.1-shade.jar' dose not hive classes related to htrace. Did I miss any configurations ?

You haven't missed any configuration. The current workaround solution is to copy hbase-noop-htrace-4.1.1.jar to worker-server/libs.

Before I saw this PR, I made a soft link (cd worker-server/libs; ln -s ../../libs/hbase-noop-htrace-4.1.1.jar hbase-noop-htrace-4.1.1.jar), it did work.