可實現的全局唯一有序ID生成策略

  • 2019 年 10 月 25 日
  • 筆記

在部落格園搜素全局唯一有序ID，羅列出來的文章大致講述了以下幾個問題，常見的生成全局唯一id的常見方法 ：使用資料庫自動增長序列實現 ； 使用UUID實現；  使用redis實現； 使用Twitter的snowflake演算法實現；使用資料庫+本地快取實現。作為一個記錄性質的部落格，簡單總結一下。

在實際的生產場景中，經常會出現如下的情況比方說訂單號：D channelNo 流水號 樣例PSDK1600000001, PSDK1600000002, PSDK1600000003... 這種具有業務意義的全局唯一id且有序自增。先來看一下使用比較多的Twitter的snowflake演算法，snowflake生成的ID整體上按照時間自增排序，並且整個分散式系統內不會產生ID碰撞（由datacenter和workerId作區分），並且效率較高。經測試snowflake每秒能夠產生26萬個ID。一個簡單的實現如下：

/**   * Twitter的分散式自增ID雪花演算法snowflake   **/  public class SnowFlake {      /**       * 起始的時間戳       */      private final static long START_STMP = 1480166465631L;      /**       * 每一部分佔用的位數       */      private final static long SEQUENCE_BIT = 12; //序列號佔用的位數      private final static long MACHINE_BIT = 5;   //機器標識佔用的位數      private final static long DATACENTER_BIT = 5;//數據中心佔用的位數      /**       * 每一部分的最大值       */      private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);      private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);      private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);      /**       * 每一部分向左的位移       */      private final static long MACHINE_LEFT = SEQUENCE_BIT;      private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;      private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;        private long datacenterId;  //數據中心      private long machineId;     //機器標識      private long sequence = 0L; //序列號      private long lastStmp = -1L;//上一次時間戳        public SnowFlake(long datacenterId, long machineId) {          if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {              throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");          }          if (machineId > MAX_MACHINE_NUM || machineId < 0) {              throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");          }          this.datacenterId = datacenterId;          this.machineId = machineId;      }        /**       * 產生下一個ID       *       * @return       */      public synchronized long nextId() {          long currStmp = getNewstmp();          if (currStmp < lastStmp) {              throw new RuntimeException("Clock moved backwards.  Refusing to generate id");          }            if (currStmp == lastStmp) {              //相同毫秒內，序列號自增              sequence = (sequence + 1) & MAX_SEQUENCE;              //同一毫秒的序列數已經達到最大              if (sequence == 0L) {                  currStmp = getNextMill();              }          } else {              //不同毫秒內，序列號置為0              sequence = 0L;          }            lastStmp = currStmp;            return (currStmp - START_STMP) << TIMESTMP_LEFT //時間戳部分                  | datacenterId << DATACENTER_LEFT       //數據中心部分                  | machineId << MACHINE_LEFT             //機器標識部分                  | sequence;                             //序列號部分      }        private long getNextMill() {          long mill = getNewstmp();          while (mill <= lastStmp) {              mill = getNewstmp();          }          return mill;      }        private long getNewstmp() {          return System.currentTimeMillis();      }        public static void main(String[] args) {          SnowFlake snowFlake = new SnowFlake(1, 1);          long start = System.currentTimeMillis();          for (int i = 0; i < 10; i++) {              System.out.println(snowFlake.nextId());          }          System.out.println(System.currentTimeMillis() - start);      }  }

演算法中引入了時間因子，所以可以保證生成的id唯一且有序，但是滿足不了業務欄位+流水號有序自增的要求。如果在此基礎上再配合使用資料庫本地快取自然也是可以實現的，不過複雜化了。上述程式碼執行兩次結果如下： 

385063405393940480  385063405393940481  385063405393940482  385063405393940483  385063405393940484  385063405393940485  385063405393940486  385063405393940487  385063405398134784  385063405398134785    385064572152844288  385064572152844289  385064572152844290  385064572152844291  385064572152844292  385064572152844293  385064572152844294  385064572152844295  385064572152844296  385064572152844297

 

簡單的方法就是我們放棄自己造輪子的思想。mongodb中數據的基本單元稱為document，在一個特定集合內部需要唯一的標識文檔，因此mongdb中存儲的文檔都由一個‘_id’鍵，這個鍵的值可以是任意類型的ObjectId，要求不同的機器都能用全局唯一的同種方法方便的生成它。因此不能使用自增主鍵，ObjectId 底層也是借鑒了雪花演算法，使用12位元組的存儲空間  |0|1|2|3|4|5|6 |7|8|9|10|11|  |時間戳  |機器ID|PID|計數器|  前四個位元組時間戳是從標準紀元開始的時間戳，單位為秒 。時間戳保證秒級唯一，機器ID保證設計時考慮分散式，避免時鐘同步，PID保證同一台伺服器運行多個mongod實例時的唯一性，最後的計數器保證同一秒內的唯一性。

 

 

 

mongo在spring boot中的引入和配置，此處不再介紹。

創建model類

package com.slowcity.admin.generate.dbmodel;    import java.io.Serializable;    public class BaseSequence implements Serializable{        private static final long serialVersionUID = 475722757687764546L;      private String id;      private String name;      private Long sequence;        public String getId() {          return id;      }      public void setId(String id) {          this.id = id;      }      public String getName() {          return name;      }      public void setName(String name) {          this.name = name;      }      public Long getSequence() {          return sequence;      }      public void setSequence(Long sequence) {          this.sequence = sequence;      }      @Override      public int hashCode() {          final int prime = 31;          int result = 1;          result = prime * result + ((id == null) ? 0 : id.hashCode());          result = prime * result + ((name == null) ? 0 : name.hashCode());          result = prime * result + ((sequence == null) ? 0 : sequence.hashCode());          return result;      }      @Override      public boolean equals(Object obj) {          if (this == obj)              return true;          if (obj == null)              return false;          if (getClass() != obj.getClass())              return false;          BaseSequence other = (BaseSequence) obj;          if (id == null) {              if (other.id != null)                  return false;          } else if (!id.equals(other.id))              return false;          if (name == null) {              if (other.name != null)                  return false;          } else if (!name.equals(other.name))              return false;          if (sequence == null) {              if (other.sequence != null)                  return false;          } else if (!sequence.equals(other.sequence))              return false;          return true;      }      @Override      public String toString() {          return "BaseSequence [id=" + id + ", name=" + name + ", sequence=" + sequence + "]";      }    }
public class DigitalTaskSequence extends BaseSequence{      private static final long serialVersionUID = -7287622688931253780L;  }
import org.springframework.data.annotation.Id;  import org.springframework.data.mongodb.core.mapping.Document;  import org.springframework.stereotype.Component;      @Component  @Document(collection = "dm_id_task")  public class DigitalTaskSequenceMG extends DigitalTaskSequence {      private static final long serialVersionUID = -425011291271386371L;      @Id      @Override      public String getId() {          return super.getId();      }  }

service

import java.util.List;    import com.slowcity.admin.generate.dbmodel.BaseSequence;    public interface SequenceGenericService {      public String generateId(Class<? extends BaseSequence> clazz);      List<BaseSequence> initAllId();  }
import java.util.ArrayList;  import java.util.List;    import org.slf4j.Logger;  import org.slf4j.LoggerFactory;  import org.springframework.stereotype.Service;  import org.springframework.transaction.annotation.Transactional;    import com.slowcity.admin.generate.dbmodel.BaseSequence;  import com.slowcity.admin.generate.dbmodel.DigitalTaskSequenceMG;  import com.slowcity.admin.generate.repository.SequenceGenericRepository;  import com.slowcity.admin.generate.service.SequenceGenericService;      @Service  @Transactional  public class SequenceGenericServiceImpl implements SequenceGenericService {      private static final Logger log = LoggerFactory.getLogger(SequenceGenericServiceImpl.class);      private SequenceGenericRepository sequenceGenericRepository;        public SequenceGenericServiceImpl(SequenceGenericRepository sequenceGenericRepository) {          this.sequenceGenericRepository = sequenceGenericRepository;      }        @Override      public String generateId(Class<? extends BaseSequence> clazz) {          String id = sequenceGenericRepository.generateId(clazz);          log.info("{} generate {}", clazz.getName(), id);          return id;      }        @Override      public List<BaseSequence> initAllId() {            List<BaseSequence> baseSequenceList = new ArrayList<>(),          baseSequenceResultList = new ArrayList<>();          DigitalTaskSequenceMG digitalTaskSequenceMG = new DigitalTaskSequenceMG();          digitalTaskSequenceMG.setName("sequence");          digitalTaskSequenceMG.setSequence(1210000000000000000L); //1210可以代表業務號 000000000000000代表自增流水號
          baseSequenceList.add(digitalTaskSequenceMG);            for (BaseSequence baseSequence:baseSequenceList) {              BaseSequence resultSequence = sequenceGenericRepository.initAllId(baseSequence);              if(resultSequence != null){                  baseSequenceResultList.add(resultSequence);              }          }          return baseSequenceResultList;      }      }

數據實現層

import com.slowcity.admin.generate.dbmodel.BaseSequence;    public interface SequenceGenericRepository {        public String generateId(Class<? extends BaseSequence> clazz);      BaseSequence initAllId(BaseSequence Sequence);    }
import java.util.List;  import java.util.Map;  import java.util.stream.Collectors;    import org.springframework.data.mongodb.core.MongoTemplate;  import org.springframework.data.mongodb.core.query.Criteria;  import org.springframework.data.mongodb.core.query.Query;  import org.springframework.data.mongodb.core.query.Update;  import org.springframework.stereotype.Component;    import com.slowcity.admin.generate.dbmodel.BaseSequence;  import com.slowcity.admin.generate.repository.SequenceGenericRepository;    @Component  public class SequenceMongoGenericRepository implements SequenceGenericRepository {      private Map<Class,Class<? extends BaseSequence>> baseSequenceMap;      private MongoTemplate mongoTemplate;      public SequenceMongoGenericRepository(List<BaseSequence> baseSequences, MongoTemplate mongoTemplate){          baseSequenceMap = baseSequences.stream()              .collect(Collectors.toMap(baseSequence -> baseSequence.getClass().getSuperclass(),                  BaseSequence::getClass));          this.mongoTemplate = mongoTemplate;      }          @Override      public String generateId(Class<? extends BaseSequence> clazz) {          Class<? extends BaseSequence> childClazz = baseSequenceMap.get(clazz);          if(childClazz != null) {              Query query = new Query(Criteria.where("name").is("sequence"));              Update update = new Update().inc("sequence", 1);              Object dbm = mongoTemplate.findAndModify(query, update, childClazz);              if(dbm != null) {                  BaseSequence bs = (BaseSequence)dbm;                  return String.valueOf(bs.getSequence());              }          }          return null;      }        @Override      public BaseSequence initAllId(BaseSequence Sequence) {            Query query = new Query(Criteria.where("name").is("sequence"));          Class clazz = Sequence.getClass();          List<? extends BaseSequence> list = mongoTemplate.find(query,clazz);          if(list.isEmpty()){              mongoTemplate.save(Sequence);              return Sequence;          }              return null;      }  }

controller

import java.util.List;    import org.springframework.beans.factory.annotation.Autowired;  import org.springframework.web.bind.annotation.RequestMapping;  import org.springframework.web.bind.annotation.RequestMethod;  import org.springframework.web.bind.annotation.RestController;    import com.slowcity.admin.generate.dbmodel.BaseSequence;  import com.slowcity.admin.generate.dbmodel.DigitalTaskSequence;  import com.slowcity.admin.generate.service.SequenceGenericService;    /**   * id生成器   * @author moona   *   */  @RestController  @RequestMapping("/generateId/task")  public class TaskGenerateIdController {        @Autowired      private SequenceGenericService sequenceGenericService;      @RequestMapping(value = "/taskId", method = RequestMethod.GET)      public String generateTaskId() {          return sequenceGenericService.generateId(DigitalTaskSequence.class);      }        @RequestMapping(value = "/init", method = RequestMethod.GET)      public  List<BaseSequence> generateTaskIdinit() {          return sequenceGenericService.initAllId();      }    }

 

執行初始化調用方法

 

 對應資料庫

 

 

開始測試生成id

第一次調用：

第2次調用

 

 第10次調用

 

 此時再查看資料庫，序列已經到1210000000000000011 下次調用直接取值了。真正做到了了分散式滿足業務的自增全局唯一索引。mongo底層是原子性的，所以也不會出現並發的問題。如果將id生成策略部署成單台機器服務，則可以滿足不同服務不同業務的需求，真正做到可訂製可擴展。盡可放心使用。

 

 

【end】

 

Tags:

VirMach 便宜 VPS