WebRTC本地選擇codec（web本地模擬）

2022 年 1 月 13 日
筆記
WEB

影片編碼後，再進行發送。WebRTC建立影片連接前，可以選擇codec。一般來說支援多種codec，以VP8和H264為代表。

Codec：編碼解碼器，編解碼器

示例程式碼

寫一個示例，用戶可以在發送影片流之前選擇codec。把支援的codec類型列出來，用戶自行選擇。

html

放上2個video控制項顯示收發影片。按鈕控制開始，呼叫（發起連接）和掛斷。

<div id="container">
    <h1><a href="//an.rustfisher.com/webrtc/peerconnection/change-codec/" title="WebRTC示例，修改codec">WebRTC示例，修改codec</a>
    </h1>

    <video id="localVideo" playsinline autoplay muted></video>
    <video id="remoteVideo" playsinline autoplay></video>

    <div class="box">
        <button id="startBtn">開始</button>
        <button id="callBtn">呼叫</button>
        <button id="hangupBtn">掛斷</button>
    </div>

    <div class="box">
        <span>選擇Codec:</span>
        <select id="codecPreferences" disabled>
            <option selected value="">Default</option>
        </select>
        <div id="actualCodec"></div>
    </div>
    <p>可以在控制台觀察 <code>MediaStream</code>， <code>localStream</code>， 和 <code>RTCPeerConnection</code></p>
</div>

<script src="../../src/js/adapter-2021.js"></script>
<script src="js/main.js" async></script>

select用來選擇codec。獲取支援的codec資訊，放到下拉欄里讓用戶選擇。

adapter-2021.js是存放在本地的文件。要使用最新的adapter，按以下地址引入

<script src="//webrtc.github.io/adapter/adapter-latest.js"></script>

js

main.js控制主要邏輯。從開啟攝影機開始。建立連接前可以選擇codec。

建立連接的流程與「WebRTC模擬傳輸影片流，video通過本地節點peer傳輸影片流」類似

main.js完整程式碼如下


'use strict';

console.log('WebRTC示例，選擇codec');

// --------- ui準備 ---------
const startBtn = document.getElementById('startBtn');
const callBtn = document.getElementById('callBtn');
const hangupBtn = document.getElementById('hangupBtn');
const localVideo = document.getElementById('localVideo');
const remoteVideo = document.getElementById('remoteVideo');

callBtn.disabled = true;
hangupBtn.disabled = true;
startBtn.addEventListener('click', start);
callBtn.addEventListener('click', call);
hangupBtn.addEventListener('click', hangup);
// ---------------------------

// -------- codec 的配置 --------
const codecPreferences = document.querySelector('#codecPreferences');
const supportsSetCodecPreferences = window.RTCRtpTransceiver &&
  'setCodecPreferences' in window.RTCRtpTransceiver.prototype;
// -----------------------------

let startTime;

remoteVideo.addEventListener('resize', () => {
  console.log(`Remote video size changed to ${remoteVideo.videoWidth}x${remoteVideo.videoHeight}`);
  if (startTime) {
    const elapsedTime = window.performance.now() - startTime;
    console.log('影片流連接耗時: ' + elapsedTime.toFixed(3) + 'ms');
    startTime = null;
  }
});

let localStream;
let pc1;
let pc2;
const offerOptions = {
  offerToReceiveAudio: 1,
  offerToReceiveVideo: 1
};

function getName(pc) {
  return (pc === pc1) ? 'pc1' : 'pc2';
}

function getOtherPc(pc) {
  return (pc === pc1) ? pc2 : pc1;
}

// 啟動本地影片
async function start() {
  console.log('啟動本地影片');
  startBtn.disabled = true;
  try {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: true });
    console.log('獲取到本地影片');
    localVideo.srcObject = stream;
    localStream = stream;
    callBtn.disabled = false;
  } catch (e) {
    alert(`getUserMedia() error: ${e.name}`);
  }
  if (supportsSetCodecPreferences) {
    const { codecs } = RTCRtpSender.getCapabilities('video');
    console.log('RTCRtpSender.getCapabilities(video):\n', codecs);
    codecs.forEach(codec => {
      if (['video/red', 'video/ulpfec', 'video/rtx'].includes(codec.mimeType)) {
        return;
      }
      const option = document.createElement('option');
      option.value = (codec.mimeType + ' ' + (codec.sdpFmtpLine || '')).trim();
      option.innerText = option.value;
      codecPreferences.appendChild(option);
    });
    codecPreferences.disabled = false;
  } else {
    console.warn('當前不支援更換codec');
  }
}

// 呼叫並建立連接
async function call() {
  callBtn.disabled = true;
  hangupBtn.disabled = false;
  console.log('開始呼叫');
  startTime = window.performance.now();
  const videoTracks = localStream.getVideoTracks();
  const audioTracks = localStream.getAudioTracks();
  if (videoTracks.length > 0) {
    console.log(`使用的攝影機: ${videoTracks[0].label}`);
  }
  if (audioTracks.length > 0) {
    console.log(`使用的麥克風: ${audioTracks[0].label}`);
  }
  const configuration = {};
  pc1 = new RTCPeerConnection(configuration);
  pc1.addEventListener('icecandidate', e => onIceCandidate(pc1, e));
  pc2 = new RTCPeerConnection(configuration);
  pc2.addEventListener('icecandidate', e => onIceCandidate(pc2, e));
  pc2.addEventListener('track', gotRemoteStream);

  localStream.getTracks().forEach(track => pc1.addTrack(track, localStream));
  if (supportsSetCodecPreferences) {
    // 獲取選擇的codec
    const preferredCodec = codecPreferences.options[codecPreferences.selectedIndex];
    if (preferredCodec.value !== '') {
      const [mimeType, sdpFmtpLine] = preferredCodec.value.split(' ');
      const { codecs } = RTCRtpSender.getCapabilities('video');
      const selectedCodecIndex = codecs.findIndex(c => c.mimeType === mimeType && c.sdpFmtpLine === sdpFmtpLine);
      const selectedCodec = codecs[selectedCodecIndex];
      codecs.splice(selectedCodecIndex, 1);
      codecs.unshift(selectedCodec);
      console.log(codecs);
      const transceiver = pc1.getTransceivers().find(t => t.sender && t.sender.track === localStream.getVideoTracks()[0]);
      transceiver.setCodecPreferences(codecs);
      console.log('選擇的codec', selectedCodec);
    }
  }
  codecPreferences.disabled = true;

  try {
    const offer = await pc1.createOffer(offerOptions);
    await onCreateOfferSuccess(offer);
  } catch (e) {
    console.log(`Failed, pc1 createOffer: ${e.toString()}`);
  }
}

async function onCreateOfferSuccess(desc) {
  try {
    await pc1.setLocalDescription(desc);
    console.log('pc1 setLocalDescription 成功');
  } catch (e) {
    console.error('pc1 setLocalDescription 出錯', e);
  }
  try {
    await pc2.setRemoteDescription(desc);
    console.log('pc2 setRemoteDescription ok');
  } catch (e) {
    console.error('pc2 setRemoteDescription fail', e);
  }
  try {
    const answer = await pc2.createAnswer();
    await onCreateAnswerSuccess(answer);
  } catch (e) {
    console.log(`pc2 create answer fail: ${e.toString()}`);
  }
}

function gotRemoteStream(e) {
  if (remoteVideo.srcObject !== e.streams[0]) {
    remoteVideo.srcObject = e.streams[0];
    console.log('pc2 received remote stream');
  }
}

// 應答（接收）成功
async function onCreateAnswerSuccess(desc) {
  console.log(`Answer from pc2:\n${desc.sdp}`);
  console.log('pc2 setLocalDescription start');
  try {
    await pc2.setLocalDescription(desc);
  } catch (e) {
    console.error('pc2 set local d fail', e);
  }
  console.log('pc1 setRemoteDescription start');
  try {
    await pc1.setRemoteDescription(desc);

    // Display the video codec that is actually used.
    setTimeout(async () => {
      const stats = await pc1.getStats();
      stats.forEach(stat => {
        if (!(stat.type === 'outbound-rtp' && stat.kind === 'video')) {
          return;
        }
        const codec = stats.get(stat.codecId);
        document.getElementById('actualCodec').innerText = 'Using ' + codec.mimeType +
          ' ' + (codec.sdpFmtpLine ? codec.sdpFmtpLine + ' ' : '') +
          ', payloadType=' + codec.payloadType + '. Encoder: ' + stat.encoderImplementation;
      });
    }, 1000);
  } catch (e) {
    console.error(e);
  }
}

async function onIceCandidate(pc, event) {
  try {
    await (getOtherPc(pc).addIceCandidate(event.candidate));
    onAddIceCandidateSuccess(pc);
  } catch (e) {
    onAddIceCandidateError(pc, e);
  }
  console.log(`${getName(pc)} ICE candidate:\n${event.candidate ? event.candidate.candidate : '(null)'}`);
}

function onAddIceCandidateSuccess(pc) {
  console.log(`${getName(pc)} addIceCandidate success`);
}

function onAddIceCandidateError(pc, error) {
  console.log(`${getName(pc)} failed to add ICE Candidate: ${error.toString()}`);
}

localVideo.addEventListener('loadedmetadata', function () {
  console.log(`Local video videoWidth: ${this.videoWidth}px,  videoHeight: ${this.videoHeight}px`);
});

remoteVideo.addEventListener('loadedmetadata', function () {
  console.log(`Remote video videoWidth: ${this.videoWidth}px,  videoHeight: ${this.videoHeight}px`);
});

// 掛斷
function hangup() {
  console.log('掛斷');
  pc1.close();
  pc2.close();
  pc1 = null;
  pc2 = null;
  hangupBtn.disabled = true;
  callBtn.disabled = false;
  codecPreferences.disabled = false;
}

獲取可用codec

先判斷瀏覽器是否有RTCRtpTransceiver，並且要能支援setCodecPreferences方法

const supportsSetCodecPreferences = window.RTCRtpTransceiver &&
  'setCodecPreferences' in window.RTCRtpTransceiver.prototype;

通過RTCRtpSender.getCapabilities('video')獲取可支援的codec。
然後把它們放進列表codecPreferences里

  if (supportsSetCodecPreferences) {
    const { codecs } = RTCRtpSender.getCapabilities('video');

    codecs.forEach(codec => {
      if (['video/red', 'video/ulpfec', 'video/rtx'].includes(codec.mimeType)) {
        return;
      }
      const option = document.createElement('option');
      option.value = (codec.mimeType + ' ' + (codec.sdpFmtpLine || '')).trim();
      option.innerText = option.value;
      codecPreferences.appendChild(option);
    });
    codecPreferences.disabled = false;
  }

呼叫的時候，把選中的codec交給transceiver

  if (supportsSetCodecPreferences) {
    // 獲取選擇的codec
    const preferredCodec = codecPreferences.options[codecPreferences.selectedIndex];
    if (preferredCodec.value !== '') {
      const [mimeType, sdpFmtpLine] = preferredCodec.value.split(' ');
      const { codecs } = RTCRtpSender.getCapabilities('video');
      const selectedCodecIndex = codecs.findIndex(c => c.mimeType === mimeType && c.sdpFmtpLine === sdpFmtpLine);
      const selectedCodec = codecs[selectedCodecIndex];
      codecs.splice(selectedCodecIndex, 1);
      codecs.unshift(selectedCodec);
      console.log(codecs);
      const transceiver = pc1.getTransceivers().find(t => t.sender && t.sender.track === localStream.getVideoTracks()[0]);
      transceiver.setCodecPreferences(codecs);
      console.log('選擇的codec', selectedCodec);
    }
  }

codec

觀察控制台的log，列印出了可用codec資訊。

0: {clockRate: 90000, mimeType: 'video/VP8'}
1: {clockRate: 90000, mimeType: 'video/rtx'}
2: {clockRate: 90000, mimeType: 'video/VP9', sdpFmtpLine: 'profile-id=0'}
3: {clockRate: 90000, mimeType: 'video/VP9', sdpFmtpLine: 'profile-id=2'}
4: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f'}
5: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f'}
6: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f'}
7: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f'}
8: {clockRate: 90000, mimeType: 'video/AV1'}
9: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0032'}
10: {clockRate: 90000, mimeType: 'video/H264', sdpFmtpLine: 'level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640032'}
11: {clockRate: 90000, mimeType: 'video/red'}
12: {clockRate: 90000, mimeType: 'video/ulpfec'}

clockRate 是codec的時鐘頻率，單位hz

sdpFmtpLine 是codec的SDP里a=fmtp的參數資訊

mimeType 里說的是影片編碼類型，常見的有VP8和H264等等。下面是影片格式的一些介紹。

VP8 VP9

2010年5月Google收購了On2 Technologies，獲得了VP8。
Opera，FireFox，Chrome和Chromium支援HTML5中的video播放VP8影片。

WebM作為一個容器格式，影像部分使用VP8，音頻使用Vorbis和Opus。

VP9由Google開發，一個開放的無版權費的影片編碼標準。開發初期曾用名「Next Gen Open Video」。VP9也被視為是VP8的下一代影片編碼標準。

H264

H.264，又稱為MPEG-4第10部分，高級影片編碼是一種面向塊，基於運動補償的影片編碼標準。
到2014年，它已經成為高精度影片錄製、壓縮和發布的最常用格式之一。

優勢：

1）網路親和性，即可適用於各種傳輸網路
2）高的影片壓縮比

目前我們用的比較多的還是H264。

效果預覽

網頁做好，效果請參考選擇codec

擴展閱讀

Tags: WEB