Skip to content

outputs为es的时候,丢数据问题 #42

Description

@fl911
  • 前提是配置consistency参数为true
  • es监听中afterBulk(long arg0, BulkRequest arg1, Throwable arg2)方法:
public void afterBulk(long arg0, BulkRequest arg1, Throwable arg2) {
          logger.error("bulk got exception:", arg2);
                        
          for(ActionRequest request : arg1.requests()){
               addFailedMsg(request);
          }
                        
}

该方法中的addFailedMsg(request)传入的参数为ActionRequest,而 addFailedMsg(Object msg):

public void addFailedMsg(Object msg) {
        if (msg instanceof Map) {
            super.addFailedMsg(msg);
            return;
        }

        throw new IllegalArgumentException("addFailedMsg only accept Map instance");
    }

中实际处理时候做了msg是否为 Map 的判断,所以从这里进去的数据永远不会被添加到 failedMsgQueue,从而导致数据丢失

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions