JavaCV 採集攝影機和麥克風數據推流直播

2021 年 4 月 10 日
筆記
javacv

越來越覺得放棄JavaCV FFmpeg native API，直接使用JavaCV二次封裝的API開發是很明智的選擇，使用JavaCV二次封裝的API開發避免了各種記憶體操作不當引起的crash。

上一次介紹了 JavaCV 採集攝影機及桌面影片數據，這次介紹一下如何採集攝影機和麥克風數據推送到流媒體伺服器。

引入依賴

跟上一次一樣，這裡使用的還是最新的JavaCV庫（1.5.5）.

<properties>
  <javacpp.version>1.5.5</javacpp.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv-platform</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
</dependencies>

影片採集錄製

在 JavaCV 採集攝影機及桌面影片數據介紹了兩種採集攝影機數據的方法，這裡採用第一種，即使用 OpencvFrameGrabber 採集攝影機數據。

public class VideoRecorder implements Runnable {

  private static final int VIDEO_DEVICE_INDEX = 0;
  private FFmpegFrameRecorder recorder;
  private int width, height;
  public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
    this.recorder = recorder;
    this.width = width;
    this.height = height;
  }

  @Override
  public void run() {
    try {
      OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
      grabber.setImageWidth(width);
      grabber.setImageHeight(height);
      grabber.start();

      long startTS = 0, videoTS = 0;
      Frame frame = null;
      while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
        if (startTS == 0) {
          startTS = System.currentTimeMillis();
        }
        videoTS = 1000 * (System.currentTimeMillis() - startTS);
        if (videoTS > recorder.getTimestamp()) {
          recorder.setTimestamp(videoTS);
        }
        recorder.record(frame);
      }

      grabber.close();
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

音頻採集錄製

音頻採集直接使用JDK的API即可，直接讀麥克風的數據塞給錄製器，取樣率為44100，16bit，小端模式。

public class AudioRecoder implements Runnable {

  private FFmpegFrameRecorder recorder;
  private int channels;
  private int sampleRate;
  public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
    this.recorder = recorder;
    this.sampleRate = sampleRate;
    this.channels = channels;
  }

  @Override
  public void run() {
    try {
      AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
      TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
      line.open(format);
      line.start();

      int sampleRate = (int) format.getSampleRate();
      int numChannels = format.getChannels();
      byte[] buffer = new byte[sampleRate * numChannels];

      while (!Thread.interrupted()) {
        int nBytesRead = 0;
        while (nBytesRead == 0) {
          nBytesRead = line.read(buffer, 0, line.available());
        }
        int nSamplesRead = nBytesRead / 2;
        short[] samples = new short[nSamplesRead];
        ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
        ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
        recorder.recordSamples(sampleRate, numChannels, sBuff);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

}

錄製推流

這裡的錄製推流涉及音頻和影片，音頻和影片分開執行緒採集並錄製，這樣可以降低延遲，在錄製推流方面還可以通過設置錄製參數降低延遲：

recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");

啟動錄製推流器，影片採集錄製執行緒和音頻採集錄製執行緒，開始推流：

recorder.start();
Thread vt = new Thread(new VideoRecorder(recorder, width, height));
Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));
		
vt.start();
at.start();
		
vt.join();
at.join();

效果展示

可以使用VLC拉取程式的推流查看效果：

拉流的音影片流資訊：

錄製推流完整源碼可以在公眾號上獲取

=========================================================
關注公眾號「HiIT青年」發送「javacv-recoder」獲取。（如果沒有收到回復，可能是你之前取消過關注。）

關注公眾號，閱讀更多文章。

Tags: javacv