Subtitling videos using Google Video Intelligence

Introduction

Video subtitles provide a better viewing experience and also improve accessibility for persons with disabilities. Manually adding subtitles to videos, however, proves to be repetitive, boring, and a lot of work. Luckily, there's a way we can automate this. In this tutorial, we'll take a look at how to automatically add subtitles to videos using Google video intelligence, Cloudinary and Next.js.

Codesandbox

The final project can be viewed on Codesandbox.

Setup

Working Knowledge of Javascript is required. Familiarity with React, Node.js, and Next.js is also recommended although not required. Ensure you have Node.js and NPM installed in your development environment.

Create a new Next.js project by running the following command in your terminal.

1npx create-next-app video-subtitles-with-google-video-intelligence

This scaffolds a minimal Next.js project. You can check out the Next.js docs for more setup options. Proceed to open your project in your favorite code editor.

Cloudinary API Keys

Cloudinary offers a suite of APIs that allow developers to upload media, apply transformations and optimize delivery. You can get started with a free account immediately. Create a new account at Cloudinary if you do not have one then login and navigate to the console page. Here you'll find your Cloud name API Key and API Secret.

Back in your project, create a new file at the root of your project and name it .env.local. Paste the following inside.

1CLOUD_NAME=YOUR_CLOUD_NAME
2
3API_KEY=YOUR_API_KEY
4
5API_SECRET=YOUR_API_SECRET

Replace YOUR_CLOUD_NAME YOUR_API_KEY and YOUR_API_SECRET with the appropriate values that we just got from the cloudinary console page.

What we've just done here is define some environment variables. These help us to keep sensitive keys and secrets away from our codebase. Next.js has built-in support for environment variables. Read about this in the docs.

Do not check the .env.local file into source control

Google Cloud Project and credentials

The Video Intelligence API is provided by Google through the Google Cloud Platform. It contains several AI-powered features that allow for things such as face detection, label detection, video transcription, and more. Today we'll be using the video transcription feature.

If you are familiar with GCP, you can follow the quickstart guide.

Create an account if you do not already have one then navigate to the project selector page.

You then need to select an existing project or create a new one. Ensure that billing is enabled for the project. Google APIs have a free tier with a monthly limit that you can get started with. Use the APIs with caution so as not to exceed your limits. Here's how you can confirm that billing is enabled..

The next step is to enable the APIs that you will be using with that project. In our case, it's just the Video Intelligence API. Here's how to enable the Video Intelligence API.

Once you've enabled the API, you need to create a new service account. Service accounts allow our application to authenticate with google and communicate with the GCP APIs. Go to the create a new service account page and select the project you created earlier. You will need to input an appropriate name for the service account. You can use the same name we used to create our Next.js project, video-subtitles-with-google-video-intelligence

Go ahead and finish creating the account. You can leave the other options as they are. Go back to the service accounts dashboard and you'll now see your recently created service account. Under the more actions button, click on Manage keys.

Click on Add key and then on Create new key

In the pop-up dialog, make sure to choose the JSON option.

Once you're done, a .json file will be downloaded to your computer.

Add the following to the .env.local file that we created earlier.

1GCP_PROJECT_ID=YOUR_GCP_PROJECT_ID
2
3GCP_PRIVATE_KEY=YOUR_GCP_PRIVATE_KEY
4
5GCP_CLIENT_EMAIL=YOUR_GCP_CLIENT_EMAIL

Replace YOUR_GCP_PROJECT_ID,YOUR_GCP_PRIVATE_KEY and YOUR_GCP_CLIENT_EMAIL, with project_id,private_key and client_email respectively from the .json file that was downloaded above.

Dependencies

The final step in the setup is to install the required dependencies. We need google video intelligence, cloudinary, formidable, and date-fns. We'll use formidable to help us parse incoming form data, this will allow us to upload videos from the frontend. date-fns is a library of date and time utilities.

Run the following command in your terminal

1npm install cloudinary formidable date-fns @google-cloud/video-intelligence

Getting started

Create a new folder at the root of your project and call it lib. This folder will contain all our shared code. Create a file named parse-form.js under the lib folder and paste the following inside.

1// lib/parse-form.js
2
3  
4
5import  {  IncomingForm  }  from  "formidable";
6
7  
8
9/**
10
11* Parses the incoming form data.
12
13*
14
15* @param  {NextApiRequest}  req The incoming request object
16
17*/
18
19export  const  parseForm  =  (req)  =>  {
20
21return  new  Promise((resolve,  reject)  =>  {
22
23const  form  =  new  IncomingForm({ keepExtensions:  true, multiples:  true  });
24
25  
26
27form.parse(req,  (error,  fields,  files)  =>  {
28
29if  (error)  {
30
31return  reject(error);
32
33}
34
35  
36
37return  resolve({  fields,  files  });
38
39});
40
41});
42
43};

This file just sets up formidable so that we can be able to parse incoming form data. Read more in the formidable docs.

Create another file under lib and name it cloudinary.js. Paste the following code inside lib/cloudinary.js.

1// lib/cloudinary.js
2
3  
4
5// Import the v2 api and rename it to cloudinary
6
7import  {  v2  as  cloudinary  }  from  "cloudinary";
8
9  
10
11// Initialize the SDK with cloud_name, api_key, and api_secret
12
13cloudinary.config({
14
15cloud_name:  process.env.CLOUD_NAME,
16
17api_key:  process.env.API_KEY,
18
19api_secret:  process.env.API_SECRET,
20
21});
22
23  
24
25const  CLOUDINARY_FOLDER_NAME  =  "automatic-subtitles/";
26
27  
28
29/**
30
31* Get cloudinary upload
32
33*
34
35* @param  {string}  id
36
37* @returns  {Promise}
38
39*/
40
41export  const  handleGetCloudinaryUpload  =  (id)  =>  {
42
43return  cloudinary.api.resource(id,  {
44
45type:  "upload",
46
47prefix:  CLOUDINARY_FOLDER_NAME,
48
49resource_type:  "video",
50
51});
52
53};
54
55  
56
57/**
58
59* Get cloudinary uploads
60
61* @returns  {Promise}
62
63*/
64
65export  const  handleGetCloudinaryUploads  =  ()  =>  {
66
67return  cloudinary.api.resources({
68
69type:  "upload",
70
71prefix:  CLOUDINARY_FOLDER_NAME,
72
73resource_type:  "video",
74
75});
76
77};
78
79  
80
81/**
82
83* Uploads a video to cloudinary and returns the upload result
84
85*
86
87* @param  {{path: string; transformation?:TransformationOptions;publicId?: string; folder?: boolean; }}  resource
88
89*/
90
91export  const  handleCloudinaryUpload  =  (resource)  =>  {
92
93return  cloudinary.uploader.upload(resource.path,  {
94
95// Folder to store video in
96
97folder:  resource.folder  ?  CLOUDINARY_FOLDER_NAME  :  null,
98
99// Public id of video.
100
101public_id:  resource.publicId,
102
103// Type of resource
104
105resource_type:  "auto",
106
107// Transformation to apply to the video
108
109transformation:  resource.transformation,
110
111});
112
113};
114
115  
116
117/**
118
119* Deletes resources from cloudinary. Takes in an array of public ids
120
121* @param  {string[]}  ids
122
123*/
124
125export  const  handleCloudinaryDelete  =  (ids)  =>  {
126
127return  cloudinary.api.delete_resources(ids,  {
128
129resource_type:  "video",
130
131});
132
133};

This file contains all the functions we need to communicate with cloudinary. We first import the v2 API from the cloudinary SDK and rename it to cloudinary. We then initialize it by calling the config method and passing the cloud name, api key, and api secret.

CLOUDINARY_FOLDER_NAME is the folder where we'll store all our videos. This will make it easier for us to get all the uploads later.

handleGetCloudinaryUpload takes in a public id and gets a single resource from cloudinary by calling the api.resource method on the cloudinary SDK. Read more about this method in the official docs

handleGetCloudinaryUploads calls the api.resources method to get all resources uploaded to the folder that we defined under the CLOUDINARY_FOLDER_NAME variable. Read about this method in the docs

handleCloudinaryUpload takes in an object containing the path to the file we want to upload and any transformations that we want to apply to the file. It calls the uploader.upload method on the SDK. Read about this method here

handleCloudinaryDelete takes in an array of public IDs and passes them to the api.delete_resources method for deletion. Read more about this here.

Create a new file under the lib folder and name it google.js. Paste the following inside lib/google.js.

1// lib/google.js
2
3  
4
5import  {
6
7VideoIntelligenceServiceClient,
8
9protos,
10
11}  from  "@google-cloud/video-intelligence";
12
13  
14
15const  client  =  new  VideoIntelligenceServiceClient({
16
17// Google cloud platform project id
18
19projectId:  process.env.GCP_PROJECT_ID,
20
21credentials:  {
22
23client_email:  process.env.GCP_CLIENT_EMAIL,
24
25private_key:  process.env.GCP_PRIVATE_KEY.replace(/\\n/gm,  "\n"),
26
27},
28
29});
30
31  
32
33/**
34
35*
36
37* @param  {string | Uint8Array}  inputContent
38
39* @returns  {Promise<protos.google.cloud.videointelligence.v1.VideoAnnotationResults>}
40
41*/
42
43export  const  analyzeVideoTranscript  =  async  (inputContent)  =>  {
44
45// Grab the operation using array destructuring. The operation is the first object in the array.
46
47const  [operation]  =  await  client.annotateVideo({
48
49// Input content
50
51inputContent:  inputContent,
52
53// Video Intelligence features
54
55features:  ["SPEECH_TRANSCRIPTION"],
56
57// Video context settings
58
59videoContext:  {
60
61speechTranscriptionConfig:  {
62
63languageCode:  "en-US",
64
65enableAutomaticPunctuation:  true,
66
67},
68
69},
70
71});
72
73  
74
75const  [operationResult]  =  await  operation.promise();
76
77  
78
79// Gets annotations for video
80
81const  [annotations]  =  operationResult.annotationResults;
82
83  
84
85return  annotations;
86
87};

We create a new client and pass it the project id and a credentials object. Here's the different ways you can authenticate the client. The analyzeVideoTranscript takes in a string or a buffer array and then calls the client's annotateVideo method with a few options. Read more about these options in the docs. Take note of the features option. We need to tell Google what operation to run. In this case, we only pass the SPEECH_TRANSCRIPTION. Read more about this here.

We call promise() on the operation and await for the promise to be complete. We then get the operation result using Javascript's destructuring. To understand the structure of the resulting data, take a look at the official documentation. We then proceed to get the first item in the annotation results and return that.

Create a new folder called videos under pages/api. Create two files inside pages/api/videos, one called index.js and [...id].js. If you're not familiar with API routes in Next.js, have a look at this documentation. [...id].js is an example of dynamic routing in Next.js. This particular syntax is designed to catch all routes. Read about this here.

Paste the following code inside pages/api/videos/index.js

1// pages/api/videos/index.js
2
3  
4
5import  {
6
7handleCloudinaryUpload,
8
9handleGetCloudinaryUploads,
10
11}  from  "../../../lib/cloudinary";
12
13import  {  parseForm  }  from  "../../../lib/parse-form";
14
15import  {  promises  as  fs  }  from  "fs";
16
17import  {  analyzeVideoTranscript  }  from  "../../../lib/google";
18
19import  {  intervalToDuration  }  from  "date-fns";
20
21  
22
23// Custom config for our API route
24
25export  const  config  =  {
26
27api:  {
28
29bodyParser:  false,
30
31},
32
33};
34
35  
36
37/**
38
39*
40
41* @param  {NextApiRequest}  req
42
43* @param  {NextApiResponse}  res
44
45*/
46
47export  default  async  function  handler(req,  res)  {
48
49switch  (req.method)  {
50
51case  "GET":  {
52
53try  {
54
55const  result  =  await  handleGetRequest();
56
57  
58
59return  res.status(200).json({ message:  "Success",  result  });
60
61}  catch  (error)  {
62
63console.error(error);
64
65return  res.status(400).json({ message:  "Error",  error  });
66
67}
68
69}
70
71  
72
73case  "POST":  {
74
75try  {
76
77const  result  =  await  handlePostRequest(req);
78
79  
80
81return  res.status(201).json({ message:  "Success",  result  });
82
83}  catch  (error)  {
84
85console.error(error);
86
87return  res.status(400).json({ message:  "Error",  error  });
88
89}
90
91}
92
93  
94
95default:  {
96
97return  res.status(405).json({ message:  "Method Not Allowed"  });
98
99}
100
101}
102
103}
104
105  
106
107const  handleGetRequest  =  async  ()  =>  {
108
109const  uploads  =  await  handleGetCloudinaryUploads();
110
111  
112
113return  uploads;
114
115};
116
117  
118
119/**
120
121* Handles the POST request to the API route.
122
123*
124
125* @param  {NextApiRequest}  req The incoming request object
126
127*/
128
129const  handlePostRequest  =  async  (req)  =>  {
130
131// Get the form data using the parseForm function
132
133const  data  =  await  parseForm(req);
134
135  
136
137// Get the video file from the form data
138
139const  {  video  }  =  data.files;
140
141  
142
143// Read the contents of the video file
144
145const  videoFile  =  await  fs.readFile(video.filepath);
146
147  
148
149// Get the base64 encoded video file
150
151const  base64Video  =  videoFile.toString("base64");
152
153  
154
155// Analyze the video transcript using Google's video intelligence API
156
157const  annotations  =  await  analyzeVideoTranscript(base64Video);
158
159  
160
161// Map through the speech transcriptions gotten from the annotations
162
163const  allSentences  =  annotations.speechTranscriptions
164
165.map((speechTranscription)  =>  {
166
167// Map through the speech transcription's alternatives. For our case, it's just one
168
169return  speechTranscription.alternatives
170
171.map((alternative)  =>  {
172
173// Get the word segments from the speech transcription
174
175const  words  =  alternative.words  ??  [];
176
177  
178
179// Place the word segments into groups of ten
180
181const  groupOfTens  =  words.reduce((group,  word,  arr)  =>  {
182
183return  (
184
185(arr  %  10
186
187?  group[group.length  -  1].push(word)
188
189:  group.push([word]))  &&  group
190
191);
192
193},  []);
194
195  
196
197// Map through the word groups and build a sentence with the start time and end time
198
199return  groupOfTens.map((group)  =>  {
200
201// Start offset time in seconds
202
203const  startOffset  =
204
205parseInt(group[0].startTime.seconds  ??  0)  +
206
207(group[0].startTime.nanos  ??  0)  /  1000000000;
208
209  
210
211// End offset time in seconds
212
213const  endOffset  =
214
215parseInt(group[group.length  -  1].endTime.seconds  ??  0)  +
216
217(group[group.length  -  1].endTime.nanos  ??  0)  /  1000000000;
218
219  
220
221return  {
222
223startTime:  startOffset,
224
225endTime:  endOffset,
226
227sentence:  group.map((word)  =>  word.word).join("  "),
228
229};
230
231});
232
233})
234
235.flat();
236
237})
238
239.flat();
240
241  
242
243// Build the subtitle file content
244
245const  subtitleContent  =  allSentences
246
247.map((sentence,  index)  =>  {
248
249// Format the start time
250
251const  startTime  =  intervalToDuration({
252
253start:  0,
254
255end:  sentence.startTime  *  1000,
256
257});
258
259  
260
261// Format the end time
262
263const  endTime  =  intervalToDuration({
264
265start:  0,
266
267end:  sentence.endTime  *  1000,
268
269});
270
271  
272
273return  `${index  +  1}\n${startTime.hours}:${startTime.minutes}:${
274
275startTime.seconds
276
277},000 --> ${endTime.hours}:${endTime.minutes}:${endTime.seconds},000\n${
278
279sentence.sentence
280
281}`;
282
283})
284
285.join("\n\n");
286
287  
288
289const  subtitlePath  =  `public/subtitles/subtitle.srt`;
290
291  
292
293// Write the subtitle file to the filesystem
294
295await  fs.writeFile(subtitlePath,  subtitleContent);
296
297  
298
299// Upload the subtitle file to Cloudinary
300
301const  subtitleUploadResult  =  await  handleCloudinaryUpload({
302
303path:  subtitlePath,
304
305folder:  false,
306
307});
308
309  
310
311// Delete the subtitle file from the filesystem
312
313await  fs.unlink(subtitlePath);
314
315  
316
317// Upload the video file to Cloudinary and apply the subtitle file as an overlay/layer
318
319const  videoUploadResult  =  await  handleCloudinaryUpload({
320
321path:  video.filepath,
322
323folder:  true,
324
325transformation:  [
326
327{
328
329background:  "black",
330
331color:  "yellow",
332
333overlay:  {
334
335font_family:  "Arial",
336
337font_size:  "32",
338
339font_weight:  "bold",
340
341resource_type:  "subtitles",
342
343public_id:  subtitleUploadResult.public_id,
344
345},
346
347},
348
349{ flags:  "layer_apply"  },
350
351],
352
353});
354
355  
356
357return  videoUploadResult;
358
359};

This is where the magic happens. At the top, we export a custom config object. This config object tells Next.js not to use the default body parser since we'll be parsing the form data on our own. Read about custom config in API routes here. The default exported function named handler is standard for Next.js API routes. We use a switch statement to only handle GET and POST requests.

handleGetRequest gets all the uploaded resources by calling the handleGetCloudinaryUploads function that we created earlier.

handlePostRequest takes in the incoming request object. We use the parseForm method that we created in the parse-form.js file to get the form data. We then get the video file, get the base64 string and pass it to analyzeVideoTranscript. This video transcribes the video using Google Video Intelligence. The following is the structure of the data that we get back

1{
2
3segment:  {
4
5startTimeOffset:  {
6
7seconds:  string;
8
9nanos:  number;
10
11};
12
13endTimeOffset:  {
14
15seconds:  string;
16
17nanos:  number;
18
19};
20
21};
22
23speechTranscriptions:  [
24
25{
26
27alternatives:  [
28
29{
30
31transcript:  string;
32
33confidence: number;
34
35words: [
36
37{
38
39startTime:  {
40
41seconds:  string;
42
43nanos: number;
44
45};
46
47endTime: {
48
49seconds:  string;
50
51nanos: number;
52
53};
54
55word: string;
56
57}
58
59];
60
61}
62
63];
64
65languageCode: string;
66
67}
68
69];
70
71}

You can also check out some sample data here

We need to convert that to the following structure

1[
2
3{
4
5startTime:  number;
6
7endTime: number;
8
9sentence: string;
10
11}
12
13]

To achieve this we map through the annotations.speechTranscriptions, then the alternatives for each speech transcription. Google returns each word separate from its start and end time. We put those words in groups of ten so that we can form sentences with ten words. We don't want our sentences to be too long. We then join the group of words to make a sentence and flatten everything.

Next, we need to create a subtitle file. Let's have a look at the structure of a subtitles(srt) file.

1number
2
3hour:minute:second,millisecond --> hour:minute:second,millisecond
4
5sentence
6
7  
8
9number
10
11hour:minute:second,millisecond --> hour:minute:second,millisecond
12
13sentence

For example

11
2
300:01:20,000 --> 00:01:30,000
4
5This is the first frame
6
7  
8
92
10
1100:01:31,000 --> 00:01:40,000
12
13This is the second frame

We model our data into this format in the following piece of code

1// Build the subtitle file content
2
3const  subtitleContent  =  allSentences
4
5.map((sentence,  index)  =>  {
6
7// Format the start time
8
9const  startTime  =  intervalToDuration({
10
11start:  0,
12
13end:  sentence.startTime  *  1000,
14
15});
16
17  
18
19// Format the end time
20
21const  endTime  =  intervalToDuration({
22
23start:  0,
24
25end:  sentence.endTime  *  1000,
26
27});
28
29  
30
31return  `${index  +  1}\n${startTime.hours}:${startTime.minutes}:${
32
33startTime.seconds
34
35},000 --> ${endTime.hours}:${endTime.minutes}:${endTime.seconds},000\n${
36
37sentence.sentence
38
39}`;
40
41})
42
43.join("\n\n");

We then use writeFile to create a new subtitle file. We upload the subtitle file to cloudinary. After this is done we upload our video to cloudinary and apply the subtitle file as a layer. Read about how this works in the cloudinary docs.

Moving on to the [...id].js file. Paste the following inside pages/api/videos/[...id].js

1// pages/api/videos/[...id].js`
2
3  
4
5import  {  NextApiRequest,  NextApiResponse  }  from  "next";
6
7import  {
8
9handleCloudinaryDelete,
10
11handleGetCloudinaryUpload,
12
13}  from  "../../../lib/cloudinary";
14
15  
16
17/**
18
19*
20
21* @param  {NextApiRequest}  req
22
23* @param  {NextApiResponse}  res
24
25*/
26
27export  default  async  function  handler(req,  res)  {
28
29const  id  =  Array.isArray(req.query.id)
30
31?  req.query.id.join("/")
32
33:  req.query.id;
34
35  
36
37switch  (req.method)  {
38
39case  "GET":  {
40
41try  {
42
43const  result  =  await  handleGetRequest(id);
44
45  
46
47return  res.status(200).json({ message:  "Success",  result  });
48
49}  catch  (error)  {
50
51console.error(error);
52
53return  res.status(400).json({ message:  "Error",  error  });
54
55}
56
57}
58
59  
60
61case  "DELETE":  {
62
63try  {
64
65const  result  =  await  handleDeleteRequest(id);
66
67  
68
69return  res.status(200).json({ message:  "Success",  result  });
70
71}  catch  (error)  {
72
73console.error(error);
74
75return  res.status(400).json({ message:  "Error",  error  });
76
77}
78
79}
80
81  
82
83default:  {
84
85return  res.status(405).json({ message:  "Method Not Allowed"  });
86
87}
88
89}
90
91}
92
93  
94
95/**
96
97* Gets a single resource from Cloudinary.
98
99*
100
101* @param  {string}  id Public ID of the video to get
102
103*/
104
105const  handleGetRequest  =  async  (id)  =>  {
106
107const  upload  =  await  handleGetCloudinaryUpload(id);
108
109  
110
111return  upload;
112
113};
114
115  
116
117/**
118
119* Handles the DELETE request to the API route.
120
121*
122
123* @param  {string}  id Public ID of the video to delete
124
125*/
126
127const  handleDeleteRequest  =  (id)  =>  {
128
129// Delete the uploaded image from Cloudinary
130
131return  handleCloudinaryDelete([id]);
132
133};

handleGetRequest calls the handleGetCloudinaryUpload with the public id of the video and gets the uploaded video.

handleDeleteRequest just deletes the resource with the given public id.

Let's move on to the frontend. Add the following code to styles/globals.css

1:root  {
2
3--color-primary:  #ff0000;
4
5--color-primary-light:  #ff4444;
6
7}
8
9  
10
11.btn  {
12
13display:  inline-block;
14
15padding:  0.5rem  1rem;
16
17background:  var(--color-primary);
18
19color:  #ffffff;
20
21border:  none;
22
23border-radius:  0.25rem;
24
25cursor:  pointer;
26
27font-size:  1rem;
28
29font-weight:  bold;
30
31text-transform:  uppercase;
32
33text-decoration:  none;
34
35text-align:  center;
36
37transition:  all  0.2s  ease-in-out;
38
39}
40
41  
42
43.btn:hover  {
44
45background:  var(--color-primary-light);
46
47box-shadow:  0  0  0.25rem  0  rgba(0,  0,  0,  0.25);
48
49}

These are just a few styles to help us with the UI.

Create a new folder at the root of your project and name it components. This folder will hold our shared components. Create a new file under components called Layout.js and paste the following code inside.

1// components/Layout.js
2  
3
4import  Head  from  "next/head";
5
6import  Link  from  "next/link";
7
8  
9export  default  function  Layout({  children  })  {
10
11return  (
12
13<div>
14
15<Head>
16
17<title>
18
19Add subtitles to videos using google video intelligence and cloudinary
20
21</title>
22
23<meta
24
25name="description"
26
27content=" Add subtitles to videos using google video intelligence and cloudinary"
28
29/>
30
31<link rel="icon"  href="/favicon.ico"  />
32
33</Head>
34
35<nav>
36
37<ul>
38
39<li>
40
41<Link  href="/">
42
43<a className="btn">Home</a>
44
45</Link>
46
47</li>
48
49<li>
50
51<Link  href="/videos">
52
53<a className="btn">Videos</a>
54
55</Link>
56
57</li>
58
59</ul>
60
61</nav>
62
63<main>{children}</main>
64
65<style jsx>{`
66
67nav {
68
69background-color: #f0f0f0;
70
71min-height: 100px;
72
73display: flex;
74
75align-items: center;
76
77}
78
79  
80
81nav ul {
82
83list-style: none;
84
85padding: 0 32px;
86
87flex: 1;
88
89display: flex;
90
91flex-flow: row nowrap;
92
93justify-content: center;
94
95gap: 8px;
96
97}
98
99`}</style>
100
101</div>
102
103);
104
105}

We'll be wrapping our pages in this component. It allows us to have a consistent layout. Paste the following code inside pages/index.js

1// pages/index.js
2
3  
4
5import  {  useRouter  }  from  "next/router";
6
7import  {  useState  }  from  "react";
8
9import  Layout  from  "../components/Layout";
10
11  
12
13export  default  function  Home()  {
14
15const  router  =  useRouter();
16
17const  [isLoading,  setIsLoading]  =  useState(false);
18
19  
20
21const  handleFormSubmit  =  async  (event)  =>  {
22
23event.preventDefault();
24
25  
26
27try  {
28
29setIsLoading(true);
30
31  
32
33const  formData  =  new  FormData(event.target);
34
35  
36
37const  response  =  await  fetch("/api/videos",  {
38
39method:  "POST",
40
41body:  formData,
42
43});
44
45  
46
47const  data  =  await  response.json();
48
49  
50
51if  (!response.ok)  {
52
53throw  data;
54
55}
56
57  
58
59router.push("/videos");
60
61}  catch  (error)  {
62
63console.error(error);
64
65}  finally  {
66
67setIsLoading(false);
68
69}
70
71};
72
73  
74
75return  (
76
77<Layout>
78
79<div className="wrapper">
80
81<form onSubmit={handleFormSubmit}>
82
83<h2>Upload video file</h2>
84
85<div className="input-group">
86
87<label htmlFor="video">Video File</label>
88
89<input
90
91type="file"
92
93name="video"
94
95id="video"
96
97accept=".mp4,.mov,.mpeg4,.avi"
98
99multiple={false}
100
101required
102
103disabled={isLoading}
104
105/>
106
107</div>
108
109<button className="btn"  type="submit"  disabled={isLoading}>
110
111Upload
112
113</button>
114
115<button className="btn"  type="reset"  disabled={isLoading}>
116
117Cancel
118
119</button>
120
121</form>
122
123</div>
124
125<style jsx>{`
126
127div.wrapper {
128
129}
130
131  
132
133div.wrapper > form {
134
135margin: 64px auto;
136
137background-color: #fdd8d8;
138
139padding: 40px 20px;
140
141width: 60%;
142
143display: flex;
144
145flex-flow: column;
146
147gap: 8px;
148
149border-radius: 0.25rem;
150
151}
152
153  
154
155div.wrapper > form > div.input-group {
156
157display: flex;
158
159flex-flow: column;
160
161gap: 8px;
162
163}
164
165  
166
167div.wrapper > form > div.input-group > label {
168
169font-weight: bold;
170
171}
172
173  
174
175div.wrapper > form > div.input-group > input {
176
177background-color: #f5f5f5;
178
179}
180
181  
182
183div.wrapper > form > button {
184
185height: 50px;
186
187}
188
189`}</style>
190
191</Layout>
192
193);
194
195}

This is a simple page with a form for uploading the video that we want to add subtitles to. handleFormSubmit makes a POST request to /api/videos with the form data and then navigates to the /videos page upon success.

Create a new folder under the pages folder and call it videos. Create two files under pages/videos called index.js and [...id].js. Please note that this is not the same as the pages/api/videos folder. Paste the following code inside pages/videos/index.js

1// pages/videos/index.js
2
3  
4
5import  Link  from  "next/link";
6
7import  Image  from  "next/image";
8
9import  {  useCallback,  useEffect,  useState  }  from  "react";
10
11import  Layout  from  "../../components/Layout";
12
13  
14
15export  default  function  VideosPage()  {
16
17const  [isLoading,  setIsLoading]  =  useState(false);
18
19const  [videos,  setVideos]  =  useState([]);
20
21  
22
23const  getVideos  =  useCallback(async  ()  =>  {
24
25try  {
26
27setIsLoading(true);
28
29  
30
31const  response  =  await  fetch("/api/videos",  {
32
33method:  "GET",
34
35});
36
37  
38
39const  data  =  await  response.json();
40
41  
42
43if  (!response.ok)  {
44
45throw  data;
46
47}
48
49  
50
51setVideos(data.result.resources);
52
53console.log(data);
54
55}  catch  (error)  {
56
57// TODO: Show error message to the user
58
59console.error(error);
60
61}  finally  {
62
63setIsLoading(false);
64
65}
66
67},  []);
68
69  
70
71useEffect(()  =>  {
72
73getVideos();
74
75},  [getVideos]);
76
77  
78
79return  (
80
81<Layout>
82
83<div className="wrapper">
84
85<div className="videos-wrapper">
86
87{videos.map((video,  index)  =>  {
88
89const  splitVideoUrl  =  video.secure_url.split(".");
90
91  
92
93splitVideoUrl[splitVideoUrl.length  -  1]  =  "jpg";
94
95  
96
97const  thumbnail  =  splitVideoUrl.join(".");
98
99  
100
101return  (
102
103<div className="video-wrapper"  key={`video-${index}`}>
104
105<div className="thumbnail">
106
107<Image
108
109src={thumbnail}
110
111alt={video.secure_url}
112
113layout="fill"
114
115></Image>
116
117</div>
118
119<div className="actions">
120
121<Link
122
123href="/videos/[...id]"
124
125as={`/videos/${video.public_id}`}
126
127>
128
129<a>Open Video</a>
130
131</Link>
132
133</div>
134
135</div>
136
137);
138
139})}
140
141</div>
142
143</div>
144
145  
146
147{!isLoading  &&  videos.length  ===  0  ?  (
148
149<div className="no-videos">
150
151<b>No videos yet</b>
152
153<Link  href="/"  passHref>
154
155<button className="btn">Upload Video</button>
156
157</Link>
158
159</div>
160
161)  :  null}
162
163  
164
165{isLoading  ?  (
166
167<div className="loading">
168
169<b>Loading...</b>
170
171</div>
172
173)  :  null}
174
175  
176
177<style jsx>{`
178
179div.wrapper {
180
181min-height: 100vh;
182
183}
184
185  
186
187div.wrapper h1 {
188
189text-align: center;
190
191}
192
193  
194
195div.wrapper div.videos-wrapper {
196
197padding: 20px;
198
199display: flex;
200
201flex-flow: row wrap;
202
203gap: 20px;
204
205}
206
207  
208
209div.wrapper div.videos-wrapper div.video-wrapper {
210
211flex: 0 0 400px;
212
213height: 400px;
214
215}
216
217  
218
219div.wrapper div.videos-wrapper div.video-wrapper div.thumbnail {
220
221position: relative;
222
223width: 100%;
224
225height: 80%;
226
227}
228
229  
230
231div.loading,
232
233div.no-videos {
234
235height: 100vh;
236
237display: flex;
238
239flex-flow: column;
240
241justify-content: center;
242
243align-items: center;
244
245gap: 8px;
246
247}
248
249`}</style>
250
251</Layout>
252
253);
254
255}

This page will call getVideos when it renders. getVideos makes a GET request to /api/videos to get all the uploaded videos. You can read about the useCallback and useEffect react hooks from the react docs. We then show thumbnails of the videos. See here on how to generate a thumbnail of a cloudinary video.

And now for the final page. Paste the following inside pages/videos/[...id].js

1// pages/videos/[...id].js
2
3  
4
5import  {  useRouter  }  from  "next/router";
6
7import  {  useCallback,  useEffect,  useState  }  from  "react";
8
9import  Layout  from  "../../components/Layout";
10
11  
12
13export  default  function  VideoPage()  {
14
15const  router  =  useRouter();
16
17  
18
19const  id  =  Array.isArray(router.query.id)
20
21?  router.query.id.join("/")
22
23:  router.query.id;
24
25  
26
27const  [isLoading,  setIsLoading]  =  useState(false);
28
29const  [video,  setVideo]  =  useState(null);
30
31  
32
33const  getVideo  =  useCallback(async  ()  =>  {
34
35try  {
36
37setIsLoading(true);
38
39const  response  =  await  fetch(`/api/videos/${id}`,  {
40
41method:  "GET",
42
43});
44
45  
46
47const  data  =  await  response.json();
48
49  
50
51if  (!response.ok)  {
52
53throw  data;
54
55}
56
57  
58
59setVideo(data.result);
60
61console.log(data);
62
63}  catch  (error)  {
64
65// TODO: Show error message to the user
66
67console.error(error);
68
69}  finally  {
70
71setIsLoading(false);
72
73}
74
75},  [id]);
76
77  
78
79useEffect(()  =>  {
80
81getVideo();
82
83},  [getVideo]);
84
85  
86
87const  handleDownload  =  async  ()  =>  {
88
89try  {
90
91setIsLoading(true);
92
93  
94
95const  response  =  await  fetch(video.secure_url,  {});
96
97  
98
99if  (response.ok)  {
100
101const  blob  =  await  response.blob();
102
103  
104
105const  fileUrl  =  URL.createObjectURL(blob);
106
107  
108
109const  a  =  document.createElement("a");
110
111a.href  =  fileUrl;
112
113a.download  =  `${video.public_id.replace("/",  "-")}.${video.format}`;
114
115document.body.appendChild(a);
116
117a.click();
118
119a.remove();
120
121return;
122
123}
124
125  
126
127throw  await  response.json();
128
129}  catch  (error)  {
130
131// TODO: Show error message to the user
132
133console.error(error);
134
135}  finally  {
136
137setIsLoading(false);
138
139}
140
141};
142
143  
144
145const  handleDelete  =  async  ()  =>  {
146
147try  {
148
149setIsLoading(true);
150
151  
152
153const  response  =  await  fetch(`/api/videos/${id}`,  {
154
155method:  "DELETE",
156
157});
158
159  
160
161const  data  =  await  response.json();
162
163  
164
165if  (!response.ok)  {
166
167throw  data;
168
169}
170
171  
172
173router.replace("/videos");
174
175}  catch  (error)  {
176
177console.error(error);
178
179}  finally  {
180
181setIsLoading(false);
182
183}
184
185};
186
187  
188
189return  (
190
191<Layout>
192
193{video  &&  !isLoading  ?  (
194
195<div className="wrapper">
196
197<div className="video-wrapper">
198
199<video src={video.secure_url}  controls></video>
200
201<div className="actions">
202
203<button
204
205className="btn"
206
207onClick={handleDownload}
208
209disabled={isLoading}
210
211>
212
213Download
214
215</button>
216
217<button
218
219className="btn"
220
221onClick={handleDelete}
222
223disabled={isLoading}
224
225>
226
227Delete
228
229</button>
230
231</div>
232
233</div>
234
235</div>
236
237)  :  null}
238
239  
240
241{isLoading  ?  (
242
243<div className="loading">
244
245<b>Loading...</b>
246
247</div>
248
249)  :  null}
250
251  
252
253<style jsx>{`
254
255div.wrapper {
256
257}
258
259  
260
261div.wrapper > div.video-wrapper {
262
263width: 80%;
264
265margin: 20px auto;
266
267display: flex;
268
269flex-flow: column;
270
271gap: 8px;
272
273}
274
275  
276
277div.wrapper > div.video-wrapper > video {
278
279width: 100%;
280
281}
282
283  
284
285div.wrapper > div.video-wrapper > div.actions {
286
287display: flex;
288
289flex-flow: row;
290
291gap: 8px;
292
293}
294
295  
296
297div.loading {
298
299height: 100vh;
300
301display: flex;
302
303justify-content: center;
304
305align-items: center;
306
307}
308
309`}</style>
310
311</Layout>
312
313);
314
315}

getVideo makes a GET request to /api/videos/:id to get the video with the given id. handleDownload just downloads the video file. handleDelete makes a DELETE request to /api/videos/:id to delete the video with the given id.

For the final piece of the puzzle. Add the following to next.config.js.

1module.exports  =  {
2
3// ... other options
4
5images:  {
6
7domains:  ["res.cloudinary.com"],
8
9},
10
11};

This is because we're using the Image component from Next.js. We need to add the cloudinary domain so that images from that domain can be optimized. Read more about this here

You can now run your application by running the following command

1npm run dev

And that's a wrap for this tutorial. You can find the full code on my Github. Please note that this is just a simple demonstration. There's a lot of ways you could optimize your application. Have a look at Google's long running operations and Cloudinary's notifications