Introduction
Google's video intelligence API is a powerful tool for video analysis. There are many different ways that we can leverage its power to deliver more productivity. In this article, we will be exploring its Face Detection feature. We will be analyzing videos stored on Cloudinary using the Google video intelligence API and drawing bounding boxes over detected faces.
TL;DR
Here's a glimpse of what we will be going through in this article.
Log into Cloudinary and obtain API credentials needed to access their API
Log in to the Google Cloud Platform and obtain credentials that will allow us to make calls to the API.
Upload videos to Cloudinary storage.
Analyze uploaded video using Google's video intelligence API.
Show the video on a webpage.
Extract annotations from the Google video intelligence analysis.
Use HTML Canvas API to draw bounding boxes over annotated faces.
The final project can be viewed on Codesandbox.
Codesandbox
You can find the full source code on my Github repository.
Getting Started
Prerequisites
This article assumes that you have already installed Node.js and NPM on your development environment and also have a code editor ready. Check out the official Node.js website if you have not yet installed Node. You will also need a video to analyze. This can be any video with people's faces in it. For the sake of this article, I have included one here
Obtaining Cloudinary API Keys.
Cloudinary is an amazing service that offers a wide range of solutions for dealing with Media storage and Optimization. You can get started with their API for free immediately.
Head over to Cloudinary and sign in or sign up for a free account. Navigate to your cloudinary console and take note of your Cloud name
API Key
and API Secret
. These will be displayed on the top left of the console page.
Creating a Google Cloud Platform project and obtaining credentials
Google has a wide range of APIs. These are all part of the GCP and you will need a GCP project to access any one of them. Let's see how we can do that now. You might find this a bit overwhelming, especially if it's your first time. Don't be hesitant though, I will walk you through it.
There's a brief explanation on how to get started on the quickstart guide page. Create an account if you do not already have one then navigate to the project selector page. You will need to select an existing project or create a new one. The next step is to ensure that billing is activated for the selected project. Don't worry though, you won't be charged for accessing the APIs immediately. Most of the APIs, including the Video Intelligence one, have a free tier with a monthly limit. As long as you're within these limits you will not be charged. Even though it is highly unlikely to exceed those limits on a development environment, you should use the API sparingly. Confirm that billing is enabled.
Our project is now ready to use. We need to enable the APIs that we will be using. In our case, this is the Video Intelligence API. Once that is enabled, proceed to create a service account. This is what we will use to authenticate our application with the API. Make your way to the Create a new service account page and select the project you created earlier.
You will be required to provide a sensible name for your service account. Give it an appropriate name. For this article, I used the name face-tracking-with-next-js
.
Leave the other defaults and go-ahead to create the service account. When you navigate back to the service accounts dashboard, your new service account will now be listed. Under the more actions button, click on Manage keys.
Click on Add key and then on Create new key
In the pop-up dialog, make sure to choose the JSON option.
Once you're done, a .json
file will be downloaded to your computer. Take note of this file's location as we will be using it later.
The fun part - Implementation
This article assumes you are familiar with Next js basics. Let's create a new project. Open your terminal and run the following command in your desired project folder
1npx create-next-app
This will scaffold a basic Next.js project for you. For more advanced installation options, please take a look at the official docs. Give your project an appropriate name like face-tracking-with-next-js
. Finally, you can change the directory into your new project and we can begin the coding.
1cd face-tracking-with-next-js
Upload video to Cloudinary storage
Let's install a few dependencies first. We will need the cloudinary NPM package.
1npm install --save cloudinary
Create a folder at the root of your project and name it lib
. Inside this folder create a new file named cloudinary.js
. Inside this file, we will initialize the Cloudinary SDK and create a function to upload videos. Paste the following inside lib/cloudinary.js
1// lib/cloudinary.js2345// Import the v2 api and rename it to cloudinary67import { v2 as cloudinary } from "cloudinary";891011// Initialize the SDK with cloud_name, api_key, and api_secret1213cloudinary.config({1415cloud_name: process.env.CLOUD_NAME,1617api_key: process.env.API_KEY,1819api_secret: process.env.API_SECRET,2021});22232425export const handleCloudinaryUpload = (path) => {2627// Create and return a new Promise2829return new Promise((resolve, reject) => {3031// Use the SDK to upload media3233cloudinary.uploader.upload(3435path,3637{3839// Folder to store video in4041folder: "videos/",4243// Type of resource4445resource_type: "video",4647},4849(error, result) => {5051if (error) {5253// Reject the promise with an error if any5455return reject(error);5657}58596061// Resolve the promise with a successful result6263return resolve(result);6465}6667);6869});7071};
At the top of the file, we import the v2
API from the package we just installed and we rename it to cloudinary
. This is just for readability and you can leave it as v2
. Just remember to change it when we use it in our code. We then proceed to initialize the SDK by calling the config
method and passing to it the cloud_name
, api_key
, and api_secret
. We also have a handleCloudinaryUpload
function that will handle the uploads for us. We pass in a path to the file that we want to upload and then call the upload
method on the SDK and pass in the path
and a few options. Read more about the upload media api and options you can pass from the official documentation. We then resolve or reject a promise depending on the success or failure of the upload.
One thing you might notice is that we've assigned our cloud_name
, api_key
, and api_secret
to environment variables that we have not created yet. Let us create those now.
Create a new file called .env.local
at the root of your project. Paste the following inside this file.
1CLOUD_NAME=YOUR_CLOUD_NAME23API_KEY=YOUR_API_KEY45API_SECRET=YOUR_API_SECRET
Make sure to replace YOUR_CLOUD_NAME
YOUR_API_KEY
and YOUR_API_SECRET
with the appropriate values that we got from the Obtaining Cloudinary API Keys
section above.
Read more about Environment Variables with Next.js here
Now that we have that in place, let's create an API route where we can post our video and upload it to cloudinary. Read more about Next js api routes on the official docs if you're not familiar with this.
Create a new file called videos.js
inside the pages/api/
folder. Paste the following code inside the file :
1// pages/api/videos.js2345// Next.js API route support: https://nextjs.org/docs/api-routes/introduction67import { promises as fs } from "fs";89import { annotateVideoWithLabels } from "../../lib/google";1011import { handleCloudinaryUpload } from "../../lib/cloudinary";12131415const videosController = async (req, res) => {1617// Check the incoming HTTP method. Handle the POST request method and reject the rest.1819switch (req.method) {2021// Handle the POST request method2223case "POST": {2425try {2627const result = await handlePostRequest();28293031// Respond to the request with a status code 201(Created)3233return res.status(201).json({3435message: "Success",3637result,3839});4041} catch (error) {4243// In case of an error, respond to the request with a status code 400(Bad Request)4445return res.status(400).json({4647message: "Error",4849error,5051});5253}5455}5657// Reject other HTTP methods with a status code 4055859default: {6061return res.status(405).json({ message: "Method Not Allowed" });6263}6465}6667};68697071const handlePostRequest = async () => {7273// Path to the file you want to upload7475const pathToFile = "public/videos/people.mp4";76777879// Upload your file to cloudinary8081const uploadResult = await handleCloudinaryUpload(pathToFile);82838485// Read the file using fs. This results in a Buffer8687const file = await fs.readFile(pathToFile);88899091// Convert the file to a base64 string in preparation for analyzing the video with google's video intelligence api9293const inputContent = file.toString("base64");94959697// Analyze the video using Google's video intelligence api9899const annotations = await annotateVideoWithLabels(inputContent);100101102103// Return an object with the cloudinary upload result and the video analysis result104105return { uploadResult, annotations };106107};108109110111export default videosController;
Let's go over this. We're defining a function that will handle the HTTP requests. This is all covered in the official docs. Inside this method, we're switching the request methods and assigning the POST request a handler method called handlePostRequest
. We're then returning a failure response for all other methods since we will not be using them for this tutorial.
In the handlePostRequest
function, we define a path to the file we want to upload. For the brevity and simplicity of this tutorial, we're having a static path to a locally stored file. This is the path stored under the variable pathToFile
. You can change this path to point to your video. In the real world, you would want to have the user select a video on their computer and post it to the backend, then upload that file to cloudinary.
We then delegate the actual upload to cloudinary to the handleCloudinaryUpload
function that we created earlier inside the lib/cloudinary.js
file. Once the upload is complete, we want to analyze that same video using Google Video Intelligence. We delegate that to a function called annotateVideoWithLabels
that we haven't created yet. We will do that shortly. We then return the upload result and also the video analysis result. What we're missing at this point is the annotateVideoWithLabels
function. Let's create that.
Analyze uploaded video using Google video intelligence
First things first, we need to install the needed dependency, The video intelligence NPM package
1npm install --save @google-cloud/video-intelligence
Once we have that we need to define a few environment variables that we can reference in our code. Open the .env.local
file that we created earlier and add the below the existing variables.
1GCP_PROJECT_ID=YOUR_GCP_PROJECT_ID23GCP_PRIVATE_KEY=YOUR_GCP_PRIVATE_KEY45GCP_CLIENT_EMAIL=YOUR_GCP_CLIENT_EMAIL
Open the .json
file we downloaded in the Creating a Google Cloud Platform project and obtaining credentials
section in a text editor. Inside the credentials.json
file, you will find the appropriate values for project id, private key, and client email. Replace YOUR_GCP_PROJECT_ID
,YOUR_GCP_PRIVATE_KEY
and YOUR_GCP_CLIENT_EMAIL
with the appropriate values from the .json
file.
Careful with that file. Do not commit it into version control since anyone with this file can be able to access your GCP project.
Next, create a file called google.js
under the lib/
folder. Paste the following code inside this file.
1// lib/google.js2345import {67VideoIntelligenceServiceClient,89protos,1011} from "@google-cloud/video-intelligence";12131415const client = new VideoIntelligenceServiceClient({1617// Google cloud platform project id1819projectId: process.env.GCP_PROJECT_ID,2021credentials: {2223client_email: process.env.GCP_CLIENT_EMAIL,2425private_key: process.env.GCP_PRIVATE_KEY.replace(/\\n/gm, "\n"),2627},2829});30313233/**3435*3637* @param {string | Uint8Array} inputContent3839* @returns {Promise<protos.google.cloud.videointelligence.v1.VideoAnnotationResults>}4041*/4243export const annotateVideoWithLabels = async (inputContent) => {4445// Grab the operation using array destructuring. The operation is the first object in the array.4647const [operation] = await client.annotateVideo({4849// Input content5051inputContent: inputContent,5253// Video Intelligence features5455features: ["FACE_DETECTION"],5657// Options for context of the video being analyzed5859videoContext: {6061// Options for the label detection feature6263faceDetectionConfig: {6465includeBoundingBoxes: true,6667includeAttributes: true,6869},7071},7273});74757677const [operationResult] = await operation.promise();78798081// Gets annotations for video8283const [annotations] = operationResult.annotationResults;84858687return annotations;8889};
We first import VideoIntelligenceServiceClient
from the package we just installed and create a new client. The client takes in the project id and a credentials object containing the client's email and private key. There are many different ways of authenticating Google APIs. Have a read in the official documentation. To learn more about the method that we have used above take a look at these github docs. The reason we use this approach is to avoid having to ship our application with the sensitive .json
file we downloaded.
After we have a client ready we proceed to define the annotateVideoWithLabels
function, which will handle the video analysis. We pass a string or a buffer array to the function and then call the client's annotateVideo
method with a few options. The official documentation contains more information on this. Allow me to touch on a few of the options.
inputContent
- This is a base64 string or buffer array of your video file. If your video is hosted on Google cloud storage, you'll want to use theinputUri
field instead. Unfortunately, only Google cloud storage URLs are supported. Otherwise, you will have to use theinputContent
.features
- This is an array of the Video intelligence features that should be run on the video. Read more in the documentation. For this tutorial, we only need theFACE_DETECTION
feature which identifies people's faces.videoContext.faceDetectionConfig.includeBoundingBoxes
- This instructs the analyzer to include bounding boxes/coordinates for where the faces are located in the framevideoContext.faceDetectionConfig.includeAttributes
- This instructs the analyzer to include facial attributes e.g. smiling, wearing glasses, e.t.c. We won't really be using this for the tutorial but it's still a nice addition to have.
We wait for the operation to complete by calling promise()
on the operation and awaiting for the Promise to complete. We then get the operation result using Javascript's destructuring. To understand the structure of the resulting data, take a look at the official documentation. The structure might look like so :
1{23annotationResults: [45{67segment: {89startTimeOffset: {},1011endTimeOffset: {1213seconds: string,1415nanos: number,1617},1819},2021faceDetectionAnnotations: [2223{2425tracks: [2627{2829segment: {3031startTimeOffset: {3233seconds: string,3435nanos: number,3637},3839endTimeOffset: {4041seconds: string,4243nanos: number,4445},4647},4849timestampedObjects: [5051{5253normalizedBoundingBox: {5455top: number,5657right: number,5859bottom: number,6061},6263timeOffset: {6465seconds: string,6667nanos: number,6869},7071attributes: [7273{7475name: string,7677confidence: number,7879},8081],8283},8485],8687attributes: [8889{9091name: string,9293confidence: number,9495},9697],9899confidence: number,100101},102103],104105thumbnail: string,106107version: string,108109},110111],112113},114115],116117}
We'll only need the first item in the array. Again we use Javascript's ES6 destructuring to get the first element in the array. With this, we have all we need to upload the video to cloudinary and analyze it for faces. Now we need to show the video on the client side.
Render the video on a webpage
Open pages/index.js
and replace it with the following code.
1import Head from "next/head";23import Image from "next/image";45import { useRef, useState, MutableRefObject } from "react";6789export default function Home() {1011/**1213* This stores a reference to the video HTML Element1415* @type {MutableRefObject<HTMLVideoElement>}1617*/1819const playerRef = useRef(null);20212223/**2425* This stores a reference to the HTML Canvas2627* @type {MutableRefObject<HTMLCanvasElement>}2829*/3031const canvasRef = useRef(null);32333435const [video, setVideo] = useState();36373839const [loading, setLoading] = useState(false);40414243const handleUploadVideo = async () => {4445try {4647// Set loading to true4849setLoading(true);50515253// Make a POST request to the `api/videos/` endpoint5455const response = await fetch("/api/videos", {5657method: "post",5859});60616263const data = await response.json();64656667// Check if the response is successful6869if (response.status >= 200 && response.status < 300) {7071const result = data.result;72737475// Update our videos state with the results7677setVideo(result);7879} else {8081throw data;8283}8485} catch (error) {8687// TODO: Handle error8889console.error(error);9091} finally {9293setLoading(false);9495// Set loading to true once a response is available9697}9899};100101102103return [104105<div key="main div">106107<Head>108109<title>Face Tracking Using Google Video Intelligence</title>110111<meta112113name="description"114115content="Face Tracking Using Google Video Intelligence"116117/>118119<link rel="icon" href="/favicon.ico" />120121</Head>122123124125<header>126127<h1>Face Tracking Using Google Video Intelligence</h1>128129</header>130131<main className="container">132133<div className="wrapper">134135<div className="actions">136137<button onClick={handleUploadVideo} disabled={loading}>138139Upload140141</button>142143</div>144145<hr />146147{loading148149? [150151<div className="loading" key="loading div">152153Please be patient as the video uploads...154155</div>,156157<hr key="loading div break" />,158159]160161: null}162163164165{video ? (166167<div className="video-wrapper">168169<div className="video-container">170171<video172173width={1000}174175height={500}176177src={video.uploadResult.secure_url}178179ref={playerRef}180181onTimeUpdate={onTimeUpdate}182183></video>184185<canvas ref={canvasRef} height={500} width={1000}></canvas>186187<div className="controls">188189<button190191onClick={() => {192193playerRef.current.play();194195}}196197>198199Play200201</button>202203<button204205onClick={() => {206207playerRef.current.pause();208209}}210211>212213Pause214215</button>216217</div>218219</div>220221222223<div className="thumbnails-wrapper">224225Thumbnails226227<div className="thumbnails">228229{video.annotations.faceDetectionAnnotations.map(230231(annotation, annotationIndex) => {232233return (234235<div236237className="thumbnail"238239key={`annotation${annotationIndex}`}240241>242243<Image244245className="thumbnail-image"246247src={`data:image/jpg;base64,${annotation.thumbnail}`}248249alt="Thumbnail"250251layout="fill"252253></Image>254255</div>256257);258259}260261)}262263</div>264265</div>266267</div>268269) : (270271<div className="no-videos">272273No video yet. Get started by clicking on upload above274275</div>276277)}278279</div>280281</main>282283</div>,284285];286287}
This is just a basic React component but let's go over what's happening. Inside our component, we use useRef
hooks to store a reference to our HTML video element and another to a Canvas element, playerRef
and canvasRef
respectively. Next, we have useState
hooks that store the state for our video upload result and another for the loading/uploading state. We then define a handleUploadVideo
function that will post the video to the /api/videos
endpoint so that it can be uploaded to cloudinary and analyzed. Remember that we're only using a locally stored file. In a real-world application, you would instead want to handle the form submission and post the selected video file. Moving on to the HTML, we have an upload button that will trigger the handleUploadVideo
, a div that only shows when the loading state is true, and a div that will hold our video and canvas elements when the video state contains data. Make sure to give the video and canvas elements width and height otherwise you might run into some weird issues with the canvas API. The canvas element is placed above the video element on the Z plane. This means that it also covers the native video controls and we won't be able to play or pause the video. We get around this by adding our own play and pause buttons above the Canvas. Finally, we have a div that shows thumbnails of all the detected faces.
Extract annotations/detected faces and draw bounding boxes using Canvas
We have our video and analysis results ready. Now, all we need to do is figure out where the faces are and draw boxes over them. How do we do that? The HTML video element has an ontimeupdate
event that we can listen to. Using this, whenever the video's current time is updated, we iterate over our detected faces and check to see if the time on the detected face matches the current time of the video. If it does, we get the bounding box coordinates and use them to draw on the canvas. There are a couple of different ways you could do this to improve performance. One that comes to mind, you could iterate over detected faces and then listen to the ontimeupdate
event and draw for each face instead of drawing for all faces at once. Let's go with the former.
You will notice that in the HTML that we wrote in the pages/index.js
, our video element has an onTimeUpdate
event listener but we haven't yet defined the handler.
1{/*...*/}23<video45// ...67onTimeUpdate={onTimeUpdate}89></video>1011{/*...*/}
Let's do that now. Add the following function to our Home
component in pages/index.js
just above the handleUploadVide
function.
1const onTimeUpdate = (ev) => {23// Video element scroll height and scroll width. We use the scroll height and width instead of the video height and width because we want to ensure the dimensions match the canvas elements.45const videoHeight = playerRef.current.scrollHeight;67const videoWidth = playerRef.current.scrollWidth;891011// Get the 2d canvas context1213const ctx = canvasRef.current.getContext("2d");14151617// Whenever the video time updates make sure to clear any drawings on the canvas1819ctx.clearRect(0, 0, videoWidth, videoHeight);20212223// The video's current time2425const currentTime = playerRef.current.currentTime;26272829// Iterate over detected faces3031for (const annotation of video.annotations.faceDetectionAnnotations) {3233// Each detected face may have different tracks3435for (const track of annotation.tracks) {3637// Get the timestamps for all bounding boxes3839for (const face of track.timestampedObjects) {4041// Get the timestamp in seconds4243const timestamp =4445parseInt(face.timeOffset.seconds ?? 0) +4647(face.timeOffset.nanos ?? 0) / 1000000000;48495051// Check if the timestamp and video's current time match. We convert them to fixed-point notations of 1 decimal place5253if (timestamp.toFixed(1) == currentTime.toFixed(1)) {5455// Get the x coordinate of the origin of the bounding box5657const x = (face.normalizedBoundingBox.left || 0) * videoWidth;58596061// Get the y coordinate of the origin of the bounding box6263const y = (face.normalizedBoundingBox.top || 0) * videoHeight;64656667// Get the width of the bounding box6869const width =7071((face.normalizedBoundingBox.right || 0) -7273(face.normalizedBoundingBox.left || 0)) *7475videoWidth;76777879// Get the height of the bounding box8081const height =8283((face.normalizedBoundingBox.bottom || 0) -8485(face.normalizedBoundingBox.top || 0)) *8687videoHeight;88899091ctx.lineWidth = 4;9293ctx.strokeStyle = "#800080";9495ctx.strokeRect(x, y, width, height);9697}9899}100101}102103}104105};
Let's go over that. This function will run every time the video's time updates. We first get the video element's height and width. Remember we're getting the element's width and height and not the actual videos. This is important because we need these dimensions to match those of the canvas that is placed above the video element. Next, we get the 2d context of the canvas. This is where the drawing will happen. We also want to clear the context to prepare it for new drawings and finally get the current time of the video. Once we have all these, we iterate over the detected faces, their tracks, and then the timestamped objects for each face. The timestamped object contains the time offset where the face is in relation to the video time and also bounding boxes with left, right, top, bottom coordinates in relation to the video's x and y planes. Read about this here. A timestamp is an object with seconds
and nanos
fields. We first need to convert the nanoseconds to seconds and add up the total time in seconds. Next, we convert the timestamp(in seconds) and the current video time to fixed-point notations of 1 decimal place and check if they match. If they match, this means that the face is in the current frame of the video and we should draw a bounding box over it. We convert them to 1 decimal place so that we can get a rough estimate instead of a precise point in time. Depending on the frame rate of the video, if we use a precise point in time, the timestamp and the video's time may never match.
Once we know that a face is in the frame, we get the x and y coordinates of the face, and also the width and height of the face's bounding box. These values are in relation to the video's width and height, so we multiply with the video width for the left and right values and the video height for the top and bottom values. See this for more info. We finally proceed to set line width and a stroke style and draw a rectangle on the canvas using the coordinates/dimensions we just got.
And there we have it. Here's the full code for pages/index.js
1// pages/index.jsx2345import Head from "next/head";67import Image from "next/image";89import { useRef, useState, MutableRefObject } from "react";10111213export default function Home() {1415/**1617* @type {MutableRefObject<HTMLVideoElement>}1819*/2021const playerRef = useRef(null);22232425/**2627* @type {MutableRefObject<HTMLCanvasElement>}2829*/3031const canvasRef = useRef(null);32333435const [video, setVideo] = useState();36373839const [loading, setLoading] = useState(false);40414243const onTimeUpdate = (ev) => {4445// Video element scroll height and scroll width. We use the scroll height and width instead of the video height and width because we want to ensure the dimensions match the canvas elements.4647const videoHeight = playerRef.current.scrollHeight;4849const videoWidth = playerRef.current.scrollWidth;50515253// Get the 2d canvas context5455const ctx = canvasRef.current.getContext("2d");56575859// Whenever the video time updates make sure to clear any drawings on the canvas6061ctx.clearRect(0, 0, videoWidth, videoHeight);62636465// The video's current time6667const currentTime = playerRef.current.currentTime;68697071// Iterate over detected faces7273for (const annotation of video.annotations.faceDetectionAnnotations) {7475// Each detected face may have different tracks7677for (const track of annotation.tracks) {7879// Get the timestamps for all bounding boxes8081for (const face of track.timestampedObjects) {8283// Get the timestamp in seconds8485const timestamp =8687parseInt(face.timeOffset.seconds ?? 0) +8889(face.timeOffset.nanos ?? 0) / 1000000000;90919293// Check if the timestamp and video's current time match. We convert them to fixed-point notations of 1 decimal place9495if (timestamp.toFixed(1) == currentTime.toFixed(1)) {9697// Get the x coordinate of the origin of the bounding box9899const x = (face.normalizedBoundingBox.left || 0) * videoWidth;100101102103// Get the y coordinate of the origin of the bounding box104105const y = (face.normalizedBoundingBox.top || 0) * videoHeight;106107108109// Get the width of the bounding box110111const width =112113((face.normalizedBoundingBox.right || 0) -114115(face.normalizedBoundingBox.left || 0)) *116117videoWidth;118119120121// Get the height of the bounding box122123const height =124125((face.normalizedBoundingBox.bottom || 0) -126127(face.normalizedBoundingBox.top || 0)) *128129videoHeight;130131132133ctx.lineWidth = 4;134135ctx.strokeStyle = "#800080";136137ctx.strokeRect(x, y, width, height);138139}140141}142143}144145}146147};148149150151const handleUploadVideo = async () => {152153try {154155// Set loading to true156157setLoading(true);158159160161// Make a POST request to the `api/videos/` endpoint162163const response = await fetch("/api/videos", {164165method: "post",166167});168169170171const data = await response.json();172173174175// Check if the response is successful176177if (response.status >= 200 && response.status < 300) {178179const result = data.result;180181182183// Update our videos state with the results184185setVideo(result);186187} else {188189throw data;190191}192193} catch (error) {194195// TODO: Handle error196197console.error(error);198199} finally {200201setLoading(false);202203// Set loading to true once a response is available204205}206207};208209210211return [212213<div key="main div">214215<Head>216217<title>Face Tracking Using Google Video Intelligence</title>218219<meta220221name="description"222223content="Face Tracking Using Google Video Intelligence"224225/>226227<link rel="icon" href="/favicon.ico" />228229</Head>230231232233<header>234235<h1>Face Tracking Using Google Video Intelligence</h1>236237</header>238239<main className="container">240241<div className="wrapper">242243<div className="actions">244245<button onClick={handleUploadVideo} disabled={loading}>246247Upload248249</button>250251</div>252253<hr />254255{loading256257? [258259<div className="loading" key="loading div">260261Please be patient as the video uploads...262263</div>,264265<hr key="loading div break" />,266267]268269: null}270271272273{video ? (274275<div className="video-wrapper">276277<div className="video-container">278279<video280281width={1000}282283height={500}284285src={video.uploadResult.secure_url}286287ref={playerRef}288289onTimeUpdate={onTimeUpdate}290291></video>292293<canvas ref={canvasRef} height={500} width={1000}></canvas>294295<div className="controls">296297<button298299onClick={() => {300301playerRef.current.play();302303}}304305>306307Play308309</button>310311<button312313onClick={() => {314315playerRef.current.pause();316317}}318319>320321Pause322323</button>324325</div>326327</div>328329330331<div className="thumbnails-wrapper">332333Thumbnails334335<div className="thumbnails">336337{video.annotations.faceDetectionAnnotations.map(338339(annotation, annotationIndex) => {340341return (342343<div344345className="thumbnail"346347key={`annotation${annotationIndex}`}348349>350351<Image352353className="thumbnail-image"354355src={`data:image/jpg;base64,${annotation.thumbnail}`}356357alt="Thumbnail"358359layout="fill"360361></Image>362363</div>364365);366367}368369)}370371</div>372373</div>374375</div>376377) : (378379<div className="no-videos">380381No video yet. Get started by clicking on upload above382383</div>384385)}386387</div>388389</main>390391</div>,392393<style key="style tag" jsx="true">{`394395* {396397box-sizing: border-box;398399}400401402403header {404405height: 100px;406407background-color: purple;408409display: flex;410411justify-content: center;412413align-items: center;414415}416417418419header h1 {420421padding: 0;422423margin: 0;424425color: white;426427}428429430431.container {432433min-height: 100vh;434435background-color: white;436437}438439440441.container .wrapper {442443max-width: 1000px;444445margin: 0 auto;446447}448449450451.container .wrapper .actions {452453display: flex;454455justify-content: center;456457align-items: center;458459}460461462463.container .wrapper .actions button {464465margin: 10px;466467padding: 20px 40px;468469width: 80%;470471font-weight: bold;472473border: none;474475border-radius: 2px;476477}478479480481.container .wrapper .actions button:hover {482483background-color: purple;484485color: white;486487}488489490491.container .wrapper .video-wrapper {492493display: flex;494495flex-flow: column;496497}498499500501.container .wrapper .video-wrapper .video-container {502503position: relative;504505width: 100%;506507height: 500px;508509background: red;510511}512513514515.container .wrapper .video-wrapper .video-container video {516517position: absolute;518519object-fit: cover;520521}522523524525.container .wrapper .video-wrapper .video-container canvas {526527position: absolute;528529z-index: 1;530531}532533534535.container .wrapper .video-wrapper .video-container .controls {536537left: 0;538539bottom: 0;540541position: absolute;542543z-index: 1;544545background-color: #ffffff5b;546547width: 100%;548549height: 40px;550551display: flex;552553justify-items: center;554555align-items: center;556557}558559560561.container .wrapper .video-wrapper .video-container .controls button {562563margin: 0 5px;564565}566567568569.container .wrapper .video-wrapper .thumbnails-wrapper {570571}572573574575.container .wrapper .video-wrapper .thumbnails-wrapper .thumbnails {576577display: flex;578579flex-flow: row wrap;580581}582583584585.container586587.wrapper588589.video-wrapper590591.thumbnails-wrapper592593.thumbnails594595.thumbnail {596597position: relative;598599flex: 0 0 20%;600601height: 200px;602603border: solid;604605}606607608609.container610611.wrapper612613.video-wrapper614615.thumbnails-wrapper616617.thumbnails618619.thumbnail620621.thumbnail-image {622623width: 100%;624625height: 100%;626627}628629630631.container .wrapper .no-videos,632633.container .wrapper .loading {634635display: flex;636637justify-content: center;638639align-items: center;640641}642643`}</style>,644645];646647}
At this point, you're ready to run your application. Open your terminal and run the following at the root of your project.
1npm run dev
Conclusion
At the moment, major technology companies, such as Apple, are very interested in and adopting facial recognition technology. AI startups are becoming unicorns as well. Without a doubt, facial recognition will play an increasingly important role in society in the coming years. Regardless of privacy concerns, facial recognition makes our streets, homes, banks, and shops safer—and more efficient.