只需下载3M安装包,就能让你的手机浏览器跟踪眼球运动
请注意,虹膜跟踪不会推断人们正在看的位置,也不能提供任何形式的身份识别。在模型的文档 和随附的模型卡中,我们详细描述了模型的预期用途,局限性和公平性(与Google的AI原则保持一致)。
MediaPipe虹膜模型能够使用单个RGB摄像机实时跟踪虹膜和瞳孔的界标,而无需专用硬件。该模型还返回眼睑和眉毛区域的界标,从而能够检测到轻微的眼球运动,例如眨眼。
引入@ tensorflow / face-landmarks-detection
左上方是@ tensorflow-models / facemesh @ 0.0.4的预测,右上方是@ tensorflow-models / face-landmarks-detection @ 0.0.1的预测。虹膜界标为红色。
熟悉我们现有的 facemesh 模型的用户可以升级到新的 faceLandmarksDetection 模型,并只需进行少量代码更改,下面将对此进行详细说明。faceLandmarksDetection对facemesh进行了三项重大改进:
-
虹膜关键点检测 -
改善眼睑轮廓检测 -
改进了对旋转脸部的检测
安装
-
通过脚本标签: -
通过NPM(通过yarn,网址://yarnpkg.com/):
<script src="//cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.js"></script><script src="//cdn.jsdelivr.net/npm/@tensorflow-models/face-landmarks-detection"></script>
$ yarn add @tensorflow-models/[email protected] $ yarn add @tensorflow/[email protected]
用法
// If you are using NPM, first require the model. If you are using script tags, you can skip this step because `faceLandmarksDetection` will already be available in the global scope.const faceLandmarksDetection = require('@tensorflow-models/face-landmarks-detection');// Load the faceLandmarksDetection model assets.const model = await faceLandmarksDetection.load( faceLandmarksDetection.SupportedPackages.mediapipeFacemesh);// Pass in a video stream to the model to obtain an array of detected faces from the MediaPipe graph.// For Node users, the `estimateFaces` API also accepts a `tf.Tensor3D`, or an ImageData object.const video = document.querySelector("video");const faces = await model.estimateFaces({ input: video });
这是一个样本预测对象:
{ faceInViewConfidence: 1, boundingBox: { topLeft: [232.28, 145.26], // [x, y] bottomRight: [449.75, 308.36], }, mesh: [ [92.07, 119.49, -17.54], // [x, y, z] [91.97, 102.52, -30.54], ... ], // x,y,z positions of each facial landmark within the input space. scaledMesh: [ [322.32, 297.58, -17.54], [322.18, 263.95, -30.54] ], // Semantic groupings of x,y,z positions. annotations: { silhouette: [ [326.19, 124.72, -3.82], [351.06, 126.30, -3.00], ... ], ... }}有关API的更多详细信息, 请参阅我们的自述文件。
性能
展望未来
//storage.googleapis.com/tfjs-models/demos/face-landmarks-detection/index.html