【RocketMQ源码学习】- 3. Client 发送同步消息

  • 2019 年 11 月 12 日
  • 筆記

本文较长,代码后面给了方法简图,希望给你帮助

发送的方式

  • 同步发送

  • 异步发送

消息的类型

  • 普通消息

  • 顺序消息

  • 事务消息

发送同步消息的时序图

为了防止读者朋友嫌烦,可以看下时序图,后面我也会给出方法的简图

 

 

源码示例【发送同步消息】

调用DefaultMQProducer.send()发送同步消息

同时需要设置发送的nameSrvAddrproducerGroupName

可以设置发送的超时时间,(默认3s), msgQueueNum(默认4个), 生产端发送异步消息失败重试次数(默认2次),同步消息无重试次数

public class Producer {      public static void main(String[] args) throws MQClientException, InterruptedException {            DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");          producer.setNamesrvAddr("127.0.0.1:9876");          producer.setInstanceName("producer1");            producer.start();            for (int i = 0; i < 10; i++)              try {                  {                      Message msg = new Message("TopicTest",                          "TagA",                          "OrderID188",                              ("Hello world" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));                      // send方法                      SendResult sendResult = producer.send(msg);                      System.out.printf("%s%n", sendResult);                  }                } catch (Exception e) {                  e.printStackTrace();              }            producer.shutdown();      }  }

3. send方法内部调用sendDefaultImpl()

private SendResult sendDefaultImpl(      Message msg,      final CommunicationMode communicationMode,      final SendCallback sendCallback,      final long timeout      // + 用户来处理异常  ) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {      // + 确保服务状态是RUNNING      this.makeSureStateOK();      // + 传参判空      Validators.checkMessage(msg, this.defaultMQProducer);        final long invokeID = random.nextLong();      long beginTimestampFirst = System.currentTimeMillis();      long beginTimestampPrev = beginTimestampFirst;      long endTimestamp = beginTimestampFirst;      /**       * 根据topic的name,从本地获取tocip信息,如果本地没有就从nameserver中取,同时缓存到本地       * 包括MessageQueueList, brokeName, topic_name       */      TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());      if (topicPublishInfo != null && topicPublishInfo.ok()) {          boolean callTimeout = false;          MessageQueue mq = null;          Exception exception = null;          SendResult sendResult = null;          /**           * 生产端的重试:异步方式最大执行次数总共3次,同步1次,           * 重试针对的是brokeExceptionMQClientExceptionRemotingException返回值失败           */          int timesTotal = communicationMode == CommunicationMode.SYNC ? 1 + this.defaultMQProducer.getRetryTimesWhenSendFailed() : 1;          int times = 0;          String[] brokersSent = new String[timesTotal];          for (; times < timesTotal; times++) {              String lastBrokerName = null == mq ? null : mq.getBrokerName();              /**               * 根据topic和broke选择1个队列               * 选择策略,产生一个随机数,hash % broke中队列数,然后hash+1               * 这个随机数:是线程私有的               */              MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName);              if (mqSelected != null) {                  mq = mqSelected;                  // 在重试数组中放入broke_name                  brokersSent[times] = mq.getBrokerName();                  try {                      beginTimestampPrev = System.currentTimeMillis();                      if (times > 0) {                          // Reset topic with namespace during resend.                          // 重置topic                          msg.setTopic(this.defaultMQProducer.withNamespace(msg.getTopic()));                      }                      // 超时就break,抛出call timeout异常,这时还没有,通过socket重试                      long costTime = beginTimestampPrev - beginTimestampFirst;                      if (timeout < costTime) {                          callTimeout = true;                          break;                      }                        // 【核心,如下】调用sendKernelImpl方法,想选中的messageQueu中投递消息                      sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout - costTime);                      // 获取当前时间,控制所有步骤的时间,不超过用户设置的超时时间,或默认超时时间                      endTimestamp = System.currentTimeMillis();                        // 把这个操作时间记录到map中,key=brokerName, value=对象[包括:brokeName, currentLatency当前操作花费时间,startTimestamp开始的时间]                      this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);                      switch (communicationMode) {                          case ASYNC:                              return null;                          case ONEWAY:                              return null;                          case SYNC:                              // 当开启了重试另外一个broke时,才会失败重试                              if (sendResult.getSendStatus() != SendStatus.SEND_OK) {                                  if (this.defaultMQProducer.isRetryAnotherBrokerWhenNotStoreOK()) {                                      continue;                                  }                              }                                return sendResult;                          default:                              break;                      }                  } catch (RemotingException e) {                      // 远程调用时异常,会重试                      endTimestamp = System.currentTimeMillis();                      // 记录操作时间                      this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);                      log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);                      log.warn(msg.toString());                      exception = e;                      continue;                  } catch (MQClientException e) {                      endTimestamp = System.currentTimeMillis();                      // 记录操作时间                      this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);                      log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);                      log.warn(msg.toString());                      exception = e;                      continue;                  } catch (MQBrokerException e) {                      endTimestamp = System.currentTimeMillis();                      // 记录操作时间                      this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);                      log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);                      log.warn(msg.toString());                      exception = e;                      switch (e.getResponseCode()) {                          case ResponseCode.TOPIC_NOT_EXIST:          // topic不存在                          case ResponseCode.SERVICE_NOT_AVAILABLE:    // 服务不可用                          case ResponseCode.SYSTEM_ERROR:             // 系统错误                          case ResponseCode.NO_PERMISSION:            // 无权限                          case ResponseCode.NO_BUYER_ID:              //                          case ResponseCode.NOT_IN_CURRENT_UNIT:      // 不在集群中                              continue;                          default:                              if (sendResult != null) {                                  return sendResult;                              }                                throw e;                      }                  } catch (InterruptedException e) {                      endTimestamp = System.currentTimeMillis();                      // 记录操作时间                      this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);                      log.warn(String.format("sendKernelImpl exception, throw exception, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);                      log.warn(msg.toString());                        log.warn("sendKernelImpl exception", e);                      log.warn(msg.toString());                      throw e;                  }              } else {                  break;              }          }            if (sendResult != null) {              return sendResult;          }            String info = String.format("Send [%d] times, still failed, cost [%d]ms, Topic: %s, BrokersSent: %s",              times,              System.currentTimeMillis() - beginTimestampFirst,              msg.getTopic(),              Arrays.toString(brokersSent));            info += FAQUrl.suggestTodo(FAQUrl.SEND_MSG_FAILED);            MQClientException mqClientException = new MQClientException(info, exception);          if (callTimeout) {              throw new RemotingTooMuchRequestException("sendDefaultImpl call timeout");          }            if (exception instanceof MQBrokerException) {              mqClientException.setResponseCode(((MQBrokerException) exception).getResponseCode());          } else if (exception instanceof RemotingConnectException) {              mqClientException.setResponseCode(ClientErrorCode.CONNECT_BROKER_EXCEPTION);          } else if (exception instanceof RemotingTimeoutException) {              mqClientException.setResponseCode(ClientErrorCode.ACCESS_BROKER_TIMEOUT);          } else if (exception instanceof MQClientException) {              mqClientException.setResponseCode(ClientErrorCode.BROKER_NOT_EXIST_EXCEPTION);          }            throw mqClientException;      }        List<String> nsList = this.getmQClientFactory().getMQClientAPIImpl().getNameServerAddressList();      if (null == nsList || nsList.isEmpty()) {          throw new MQClientException(              "No name server address, please set it." + FAQUrl.suggestTodo(FAQUrl.NAME_SERVER_ADDR_NOT_EXIST_URL), null).setResponseCode(ClientErrorCode.NO_NAME_SERVER_EXCEPTION);      }        throw new MQClientException("No route info of this topic, " + msg.getTopic() + FAQUrl.suggestTodo(FAQUrl.NO_TOPIC_ROUTE_INFO),          null).setResponseCode(ClientErrorCode.NOT_FOUND_TOPIC_EXCEPTION);  }

sendDefaultImpl方法简图

  

4. 构建发送参数,使用netty发送消息[sendKernelImpl]

private SendResult sendKernelImpl(final Message msg,                                      final MessageQueue mq,                                      final CommunicationMode communicationMode,                                      final SendCallback sendCallback,                                      final TopicPublishInfo topicPublishInfo,                                      final long timeout) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {      long beginStartTime = System.currentTimeMillis();      // 获取broker的IP地址,获取到的是主broker      String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());      if (null == brokerAddr) {          // 使用topic的name去获取topic,如果本地没有,则在从nameserver中获取,同时也更新broker的信息          tryToFindTopicPublishInfo(mq.getTopic());          brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());      }        SendMessageContext context = null;      if (brokerAddr != null) {          // 若开启了vipchannel,broke的端口减2          brokerAddr = MixAll.brokerVIPChannel(this.defaultMQProducer.isSendMessageWithVIPChannel(), brokerAddr);            byte[] prevBody = msg.getBody();          try {              //for MessageBatch,ID has been set in the generating process              if (!(msg instanceof MessageBatch)) {                  // 设置UNIQ_KEY                  MessageClientIDSetter.setUniqID(msg);              }                boolean topicWithNamespace = false;              if (null != this.mQClientFactory.getClientConfig().getNamespace()) {                  msg.setInstanceId(this.mQClientFactory.getClientConfig().getNamespace());                  topicWithNamespace = true;              }                // 尝试压缩消息,有一定的条件,不是MessageBatch,消息超过4k              int sysFlag = 0;              boolean msgBodyCompressed = false;              if (this.tryToCompressMessage(msg)) {                  sysFlag |= MessageSysFlag.COMPRESSED_FLAG;                  msgBodyCompressed = true;              }                // 判断是否是事务消息              final String tranMsg = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);              if (tranMsg != null && Boolean.parseBoolean(tranMsg)) {                  sysFlag |= MessageSysFlag.TRANSACTION_PREPARED_TYPE;              }                // 禁用钩子 todo 疑问待解=>              if (hasCheckForbiddenHook()) {                  CheckForbiddenContext checkForbiddenContext = new CheckForbiddenContext();                  checkForbiddenContext.setNameSrvAddr(this.defaultMQProducer.getNamesrvAddr());                  checkForbiddenContext.setGroup(this.defaultMQProducer.getProducerGroup());                  checkForbiddenContext.setCommunicationMode(communicationMode);                  checkForbiddenContext.setBrokerAddr(brokerAddr);                  checkForbiddenContext.setMessage(msg);                  checkForbiddenContext.setMq(mq);                  checkForbiddenContext.setUnitMode(this.isUnitMode());                  this.executeCheckForbiddenHook(checkForbiddenContext);              }                // 发送消息钩子              if (this.hasSendMessageHook()) {                  context = new SendMessageContext();                  context.setProducer(this);                  context.setProducerGroup(this.defaultMQProducer.getProducerGroup());                  context.setCommunicationMode(communicationMode);                  context.setBornHost(this.defaultMQProducer.getClientIP());                  context.setBrokerAddr(brokerAddr);                  context.setMessage(msg);                  context.setMq(mq);                  context.setNamespace(this.defaultMQProducer.getNamespace());                  // 事务消息消息类型是半消息                  String isTrans = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);                  if (isTrans != null && isTrans.equals("true")) {                      context.setMsgType(MessageType.Trans_Msg_Half);                  }                    if (msg.getProperty("__STARTDELIVERTIME") != null || msg.getProperty(MessageConst.PROPERTY_DELAY_TIME_LEVEL) != null) {                      context.setMsgType(MessageType.Delay_Msg);                  }                     // 记录消息轨迹                  this.executeSendMessageHookBefore(context);              }                // 构建请求头              SendMessageRequestHeader requestHeader = new SendMessageRequestHeader();              requestHeader.setProducerGroup(this.defaultMQProducer.getProducerGroup());              requestHeader.setTopic(msg.getTopic());              requestHeader.setDefaultTopic(this.defaultMQProducer.getCreateTopicKey());              requestHeader.setDefaultTopicQueueNums(this.defaultMQProducer.getDefaultTopicQueueNums());              requestHeader.setQueueId(mq.getQueueId());              requestHeader.setSysFlag(sysFlag);              requestHeader.setBornTimestamp(System.currentTimeMillis());              requestHeader.setFlag(msg.getFlag());              requestHeader.setProperties(MessageDecoder.messageProperties2String(msg.getProperties()));              requestHeader.setReconsumeTimes(0);              requestHeader.setUnitMode(this.isUnitMode());              requestHeader.setBatch(msg instanceof MessageBatch);              // 更新 %RETRY%重试topic里消息的消费时间和 最大消费次数              if (requestHeader.getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {                  String reconsumeTimes = MessageAccessor.getReconsumeTime(msg);                  if (reconsumeTimes != null) {                      requestHeader.setReconsumeTimes(Integer.valueOf(reconsumeTimes));                      MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_RECONSUME_TIME);                  }                    String maxReconsumeTimes = MessageAccessor.getMaxReconsumeTimes(msg);                  if (maxReconsumeTimes != null) {                      requestHeader.setMaxReconsumeTimes(Integer.valueOf(maxReconsumeTimes));                      MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_MAX_RECONSUME_TIMES);                  }              }                SendResult sendResult = null;              switch (communicationMode) {                  // 异步                  case ASYNC:                      Message tmpMessage = msg;                      boolean messageCloned = false;                      if (msgBodyCompressed) {                          //If msg body was compressed, msgbody should be reset using prevBody. 如果消息体被压缩了,应该用prevBody重置msgBody                          //Clone new message using commpressed message body and recover origin massage.                          //Fix bug:https://github.com/apache/rocketmq-externals/issues/66                          tmpMessage = MessageAccessor.cloneMessage(msg);                          messageCloned = true;                          msg.setBody(prevBody);                      }                        if (topicWithNamespace) {                          if (!messageCloned) {                              tmpMessage = MessageAccessor.cloneMessage(msg);                              messageCloned = true;                          }                          msg.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic(), this.defaultMQProducer.getNamespace()));                      }                        // 防止发送之前的处理超时                      long costTimeAsync = System.currentTimeMillis() - beginStartTime;                      if (timeout < costTimeAsync) {                          throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");                      }                        // 下面仔细说明这个方法                       sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(                          brokerAddr,                          mq.getBrokerName(),                          tmpMessage,                          requestHeader,                          timeout - costTimeAsync,                          communicationMode,                          sendCallback,                          topicPublishInfo,                          this.mQClientFactory,                          this.defaultMQProducer.getRetryTimesWhenSendAsyncFailed(),                          context,                          this);                      break;                  // 直接发送,不关心发送结果/同步消息                  case ONEWAY:                  case SYNC:                      // 防止发送之前的处理超时                      long costTimeSync = System.currentTimeMillis() - beginStartTime;                      if (timeout < costTimeSync) {                          throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");                      }                      // 下面仔细说明这个方法                      sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(                          brokerAddr,                          mq.getBrokerName(),                          msg,                          requestHeader,                          timeout - costTimeSync,                          communicationMode,                          context,                          this);                      break;                  default:                      assert false;                      break;              }                if (this.hasSendMessageHook()) {                  context.setSendResult(sendResult);                  this.executeSendMessageHookAfter(context);              }                return sendResult;          } catch (RemotingException e) {              if (this.hasSendMessageHook()) {                  context.setException(e);                  this.executeSendMessageHookAfter(context);              }              throw e;          } catch (MQBrokerException e) {              if (this.hasSendMessageHook()) {                  context.setException(e);                  this.executeSendMessageHookAfter(context);              }              throw e;          } catch (InterruptedException e) {              if (this.hasSendMessageHook()) {                  context.setException(e);                  this.executeSendMessageHookAfter(context);              }              throw e;          } finally {              msg.setBody(prevBody);              msg.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic(), this.defaultMQProducer.getNamespace()));          }      }        throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null);  }

方法简图

7. 使用netty发送同步消息[invokeSync]

this.mQClientFactory.getMQClientAPIImpl().sendMessage()

内部调用

  异步消息调用的是 NettyRemotingClient.invokeAsync方法

  同步消息调用的是 NettyRemotingClient.invokeSync方法

下面跟着代码查看invokeSync方法

@Override  public RemotingCommand invokeSync(String addr, final RemotingCommand request, long timeoutMillis)      throws InterruptedException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException {      long beginStartTime = System.currentTimeMillis();      // 创建channal,如果channel为空,使用 this.bootstrap.connect(RemotingHelper.string2SocketAddress(addr));来创建channel      final Channel channel = this.getAndCreateChannel(addr);      // 活跃的channel才能发消息      if (channel != null && channel.isActive()) {          try {              // rpc鉴权              doBeforeRpcHooks(addr, request);              long costTime = System.currentTimeMillis() - beginStartTime;              if (timeoutMillis < costTime) {                  throw new RemotingTimeoutException("invokeSync call timeout");              }              //              RemotingCommand response = this.invokeSyncImpl(channel, request, timeoutMillis - costTime);              //              doAfterRpcHooks(RemotingHelper.parseChannelRemoteAddr(channel), request, response);              return response;          } catch (RemotingSendRequestException e) {              log.warn("invokeSync: send request exception, so close the channel[{}]", addr);              this.closeChannel(addr, channel);              throw e;          } catch (RemotingTimeoutException e) {              if (nettyClientConfig.isClientCloseSocketIfTimeout()) {                  this.closeChannel(addr, channel);                  log.warn("invokeSync: close socket because of timeout, {}ms, {}", timeoutMillis, addr);              }              log.warn("invokeSync: wait response timeout exception, the channel[{}]", addr);              throw e;          }      } else {          this.closeChannel(addr, channel);          throw new RemotingConnectException(addr);      }  }

 

 

8. 使用netty发送同步消息[invokeSyncImpl]

public RemotingCommand invokeSyncImpl(final Channel channel, final RemotingCommand request,      final long timeoutMillis)      throws InterruptedException, RemotingSendRequestException, RemotingTimeoutException {      final int opaque = request.getOpaque();        try {          final ResponseFuture responseFuture = new ResponseFuture(channel, opaque, timeoutMillis, null, null);          this.responseTable.put(opaque, responseFuture);          final SocketAddress addr = channel.remoteAddress();          // 使用writeAndFlush发请求,建立返回值的监听,这是netty-client发送消息,还有netty-server收消息          channel.writeAndFlush(request).addListener(new ChannelFutureListener() {              @Override              public void operationComplete(ChannelFuture f) throws Exception {                  if (f.isSuccess()) {                      responseFuture.setSendRequestOK(true);                      // 执行成功后return, 跳出这个监听                      return;                  } else {                      responseFuture.setSendRequestOK(false);                  }                    responseTable.remove(opaque);                  responseFuture.setCause(f.cause());                  responseFuture.putResponse(null);                  log.warn("send a request command to channel <" + addr + "> failed.");              }          });            // 使用countDownLatch挂起线程,等待收到netty的返回值          RemotingCommand responseCommand = responseFuture.waitResponse(timeoutMillis);          // 在clinet的channelRead0将responseCommand赋值,若为空,说明超过一定时间还未获取返回值,这时抛出异常,交由用户处理          // 可能netty-server收到消息,也有可能没有收到消息,不确定          if (null == responseCommand) {              if (responseFuture.isSendRequestOK()) {                  throw new RemotingTimeoutException(RemotingHelper.parseSocketAddressAddr(addr), timeoutMillis,                      responseFuture.getCause());              } else {                  throw new RemotingSendRequestException(RemotingHelper.parseSocketAddressAddr(addr), responseFuture.getCause());              }          }            return responseCommand;      } finally {          this.responseTable.remove(opaque);      }  }

 

 整体重要步骤

0. 用send方法发送同步消息

1. 判断发送service是否处于running状态

2. 检验发送的参数 如topic是否为空

3. 根据topicName获取topic详情,里面有这个topic的队列

4. 选择其中一个队列,选择的策略是:产生一个随机数,hash%brokeSize 然后hash+1, 同时把随机数记录下来,下次还是使用这个随机数

5. 获取broke的IP地址,如果本地没有,则从nameserver中获取

6. 如果开启了vipchannel,则端口缺口

7. 尝试压缩消息,消息数不大于4K

8. 构建消息头

9. 根据IP创建channel

10. rpc鉴权

11. 创建responseFuture, 并把他放到reponseTable中

12. 使用channel.writeAndFlush发起netty请求,并建立监听

13. 使用countDownLatch挂起线程,等待收到netty的返回值

14. 返回结果

 结语

刚开始开发送的代码,觉得太长了,不想看了,现在想想他的逻辑挺清晰的,能够帮助你看清别人的开源框架整个构建构成

======  【多学一点,for Better】======