Subtitling videos using Google Video Intelligence

Eugene Musebe

Introduction

Video subtitles provide a better viewing experience and also improve accessibility for persons with disabilities. Manually adding subtitles to videos, however, proves to be repetitive, boring, and a lot of work. Luckily, there's a way we can automate this. In this tutorial, we'll take a look at how to automatically add subtitles to videos using Google video intelligence, Cloudinary and Next.js.

Codesandbox

The final project can be viewed on Codesandbox.

Setup

Working Knowledge of Javascript is required. Familiarity with React, Node.js, and Next.js is also recommended although not required. Ensure you have Node.js and NPM installed in your development environment.

Create a new Next.js project by running the following command in your terminal.

1npx create-next-app video-subtitles-with-google-video-intelligence

This scaffolds a minimal Next.js project. You can check out the Next.js docs for more setup options. Proceed to open your project in your favorite code editor.

Cloudinary API Keys

Cloudinary offers a suite of APIs that allow developers to upload media, apply transformations and optimize delivery. You can get started with a free account immediately. Create a new account at Cloudinary if you do not have one then login and navigate to the console page. Here you'll find your Cloud name API Key and API Secret.

Back in your project, create a new file at the root of your project and name it .env.local. Paste the following inside.

1CLOUD_NAME=YOUR_CLOUD_NAME
2
3API_KEY=YOUR_API_KEY
4
5API_SECRET=YOUR_API_SECRET

Replace YOUR_CLOUD_NAME YOUR_API_KEY and YOUR_API_SECRET with the appropriate values that we just got from the cloudinary console page.

What we've just done here is define some environment variables. These help us to keep sensitive keys and secrets away from our codebase. Next.js has built-in support for environment variables. Read about this in the docs.

Do not check the .env.local file into source control

Google Cloud Project and credentials

The Video Intelligence API is provided by Google through the Google Cloud Platform. It contains several AI-powered features that allow for things such as face detection, label detection, video transcription, and more. Today we'll be using the video transcription feature.

If you are familiar with GCP, you can follow the quickstart guide.

Create an account if you do not already have one then navigate to the project selector page.

You then need to select an existing project or create a new one. Ensure that billing is enabled for the project. Google APIs have a free tier with a monthly limit that you can get started with. Use the APIs with caution so as not to exceed your limits. Here's how you can confirm that billing is enabled..

The next step is to enable the APIs that you will be using with that project. In our case, it's just the Video Intelligence API. Here's how to enable the Video Intelligence API.

Once you've enabled the API, you need to create a new service account. Service accounts allow our application to authenticate with google and communicate with the GCP APIs. Go to the create a new service account page and select the project you created earlier. You will need to input an appropriate name for the service account. You can use the same name we used to create our Next.js project, video-subtitles-with-google-video-intelligence

.

Go ahead and finish creating the account. You can leave the other options as they are. Go back to the service accounts dashboard and you'll now see your recently created service account. Under the more actions button, click on Manage keys.

Click on Add key and then on Create new key

In the pop-up dialog, make sure to choose the JSON option.

Once you're done, a .json file will be downloaded to your computer.

Add the following to the .env.local file that we created earlier.

1GCP_PROJECT_ID=YOUR_GCP_PROJECT_ID
2
3GCP_PRIVATE_KEY=YOUR_GCP_PRIVATE_KEY
4
5GCP_CLIENT_EMAIL=YOUR_GCP_CLIENT_EMAIL

Replace YOUR_GCP_PROJECT_ID,YOUR_GCP_PRIVATE_KEY and YOUR_GCP_CLIENT_EMAIL, with project_id,private_key and client_email respectively from the .json file that was downloaded above.

Dependencies

The final step in the setup is to install the required dependencies. We need google video intelligence, cloudinary, formidable, and date-fns. We'll use formidable to help us parse incoming form data, this will allow us to upload videos from the frontend. date-fns is a library of date and time utilities.

Run the following command in your terminal

1npm install cloudinary formidable date-fns @google-cloud/video-intelligence

Getting started

Create a new folder at the root of your project and call it lib. This folder will contain all our shared code. Create a file named parse-form.js under the lib folder and paste the following inside.

1// lib/parse-form.js
2
3
4
5import { IncomingForm } from "formidable";
6
7
8
9/**
10
11* Parses the incoming form data.
12
13*
14
15* @param {NextApiRequest} req The incoming request object
16
17*/
18
19export const parseForm = (req) => {
20
21return new Promise((resolve, reject) => {
22
23const form = new IncomingForm({ keepExtensions: true, multiples: true });
24
25
26
27form.parse(req, (error, fields, files) => {
28
29if (error) {
30
31return reject(error);
32
33}
34
35
36
37return resolve({ fields, files });
38
39});
40
41});
42
43};

This file just sets up formidable so that we can be able to parse incoming form data. Read more in the formidable docs.

Create another file under lib and name it cloudinary.js. Paste the following code inside lib/cloudinary.js.

1// lib/cloudinary.js
2
3
4
5// Import the v2 api and rename it to cloudinary
6
7import { v2 as cloudinary } from "cloudinary";
8
9
10
11// Initialize the SDK with cloud_name, api_key, and api_secret
12
13cloudinary.config({
14
15cloud_name: process.env.CLOUD_NAME,
16
17api_key: process.env.API_KEY,
18
19api_secret: process.env.API_SECRET,
20
21});
22
23
24
25const CLOUDINARY_FOLDER_NAME = "automatic-subtitles/";
26
27
28
29/**
30
31* Get cloudinary upload
32
33*
34
35* @param {string} id
36
37* @returns {Promise}
38
39*/
40
41export const handleGetCloudinaryUpload = (id) => {
42
43return cloudinary.api.resource(id, {
44
45type: "upload",
46
47prefix: CLOUDINARY_FOLDER_NAME,
48
49resource_type: "video",
50
51});
52
53};
54
55
56
57/**
58
59* Get cloudinary uploads
60
61* @returns {Promise}
62
63*/
64
65export const handleGetCloudinaryUploads = () => {
66
67return cloudinary.api.resources({
68
69type: "upload",
70
71prefix: CLOUDINARY_FOLDER_NAME,
72
73resource_type: "video",
74
75});
76
77};
78
79
80
81/**
82
83* Uploads a video to cloudinary and returns the upload result
84
85*
86
87* @param {{path: string; transformation?:TransformationOptions;publicId?: string; folder?: boolean; }} resource
88
89*/
90
91export const handleCloudinaryUpload = (resource) => {
92
93return cloudinary.uploader.upload(resource.path, {
94
95// Folder to store video in
96
97folder: resource.folder ? CLOUDINARY_FOLDER_NAME : null,
98
99// Public id of video.
100
101public_id: resource.publicId,
102
103// Type of resource
104
105resource_type: "auto",
106
107// Transformation to apply to the video
108
109transformation: resource.transformation,
110
111});
112
113};
114
115
116
117/**
118
119* Deletes resources from cloudinary. Takes in an array of public ids
120
121* @param {string[]} ids
122
123*/
124
125export const handleCloudinaryDelete = (ids) => {
126
127return cloudinary.api.delete_resources(ids, {
128
129resource_type: "video",
130
131});
132
133};

This file contains all the functions we need to communicate with cloudinary. We first import the v2 API from the cloudinary SDK and rename it to cloudinary. We then initialize it by calling the config method and passing the cloud name, api key, and api secret.

CLOUDINARY_FOLDER_NAME is the folder where we'll store all our videos. This will make it easier for us to get all the uploads later.

handleGetCloudinaryUpload takes in a public id and gets a single resource from cloudinary by calling the api.resource method on the cloudinary SDK. Read more about this method in the official docs

handleGetCloudinaryUploads calls the api.resources method to get all resources uploaded to the folder that we defined under the CLOUDINARY_FOLDER_NAME variable. Read about this method in the docs

handleCloudinaryUpload takes in an object containing the path to the file we want to upload and any transformations that we want to apply to the file. It calls the uploader.upload method on the SDK. Read about this method here

handleCloudinaryDelete takes in an array of public IDs and passes them to the api.delete_resources method for deletion. Read more about this here.

Create a new file under the lib folder and name it google.js. Paste the following inside lib/google.js.

1// lib/google.js
2
3
4
5import {
6
7VideoIntelligenceServiceClient,
8
9protos,
10
11} from "@google-cloud/video-intelligence";
12
13
14
15const client = new VideoIntelligenceServiceClient({
16
17// Google cloud platform project id
18
19projectId: process.env.GCP_PROJECT_ID,
20
21credentials: {
22
23client_email: process.env.GCP_CLIENT_EMAIL,
24
25private_key: process.env.GCP_PRIVATE_KEY.replace(/\\n/gm, "\n"),
26
27},
28
29});
30
31
32
33/**
34
35*
36
37* @param {string | Uint8Array} inputContent
38
39* @returns {Promise<protos.google.cloud.videointelligence.v1.VideoAnnotationResults>}
40
41*/
42
43export const analyzeVideoTranscript = async (inputContent) => {
44
45// Grab the operation using array destructuring. The operation is the first object in the array.
46
47const [operation] = await client.annotateVideo({
48
49// Input content
50
51inputContent: inputContent,
52
53// Video Intelligence features
54
55features: ["SPEECH_TRANSCRIPTION"],
56
57// Video context settings
58
59videoContext: {
60
61speechTranscriptionConfig: {
62
63languageCode: "en-US",
64
65enableAutomaticPunctuation: true,
66
67},
68
69},
70
71});
72
73
74
75const [operationResult] = await operation.promise();
76
77
78
79// Gets annotations for video
80
81const [annotations] = operationResult.annotationResults;
82
83
84
85return annotations;
86
87};

We create a new client and pass it the project id and a credentials object. Here's the different ways you can authenticate the client. The analyzeVideoTranscript takes in a string or a buffer array and then calls the client's annotateVideo method with a few options. Read more about these options in the docs. Take note of the features option. We need to tell Google what operation to run. In this case, we only pass the SPEECH_TRANSCRIPTION. Read more about this here.

We call promise() on the operation and await for the promise to be complete. We then get the operation result using Javascript's destructuring. To understand the structure of the resulting data, take a look at the official documentation. We then proceed to get the first item in the annotation results and return that.

Create a new folder called videos under pages/api. Create two files inside pages/api/videos, one called index.js and [...id].js. If you're not familiar with API routes in Next.js, have a look at this documentation. [...id].js is an example of dynamic routing in Next.js. This particular syntax is designed to catch all routes. Read about this here.

Paste the following code inside pages/api/videos/index.js

1// pages/api/videos/index.js
2
3
4
5import {
6
7handleCloudinaryUpload,
8
9handleGetCloudinaryUploads,
10
11} from "../../../lib/cloudinary";
12
13import { parseForm } from "../../../lib/parse-form";
14
15import { promises as fs } from "fs";
16
17import { analyzeVideoTranscript } from "../../../lib/google";
18
19import { intervalToDuration } from "date-fns";
20
21
22
23// Custom config for our API route
24
25export const config = {
26
27api: {
28
29bodyParser: false,
30
31},
32
33};
34
35
36
37/**
38
39*
40
41* @param {NextApiRequest} req
42
43* @param {NextApiResponse} res
44
45*/
46
47export default async function handler(req, res) {
48
49switch (req.method) {
50
51case "GET": {
52
53try {
54
55const result = await handleGetRequest();
56
57
58
59return res.status(200).json({ message: "Success", result });
60
61} catch (error) {
62
63console.error(error);
64
65return res.status(400).json({ message: "Error", error });
66
67}
68
69}
70
71
72
73case "POST": {
74
75try {
76
77const result = await handlePostRequest(req);
78
79
80
81return res.status(201).json({ message: "Success", result });
82
83} catch (error) {
84
85console.error(error);
86
87return res.status(400).json({ message: "Error", error });
88
89}
90
91}
92
93
94
95default: {
96
97return res.status(405).json({ message: "Method Not Allowed" });
98
99}
100
101}
102
103}
104
105
106
107const handleGetRequest = async () => {
108
109const uploads = await handleGetCloudinaryUploads();
110
111
112
113return uploads;
114
115};
116
117
118
119/**
120
121* Handles the POST request to the API route.
122
123*
124
125* @param {NextApiRequest} req The incoming request object
126
127*/
128
129const handlePostRequest = async (req) => {
130
131// Get the form data using the parseForm function
132
133const data = await parseForm(req);
134
135
136
137// Get the video file from the form data
138
139const { video } = data.files;
140
141
142
143// Read the contents of the video file
144
145const videoFile = await fs.readFile(video.filepath);
146
147
148
149// Get the base64 encoded video file
150
151const base64Video = videoFile.toString("base64");
152
153
154
155// Analyze the video transcript using Google's video intelligence API
156
157const annotations = await analyzeVideoTranscript(base64Video);
158
159
160
161// Map through the speech transcriptions gotten from the annotations
162
163const allSentences = annotations.speechTranscriptions
164
165.map((speechTranscription) => {
166
167// Map through the speech transcription's alternatives. For our case, it's just one
168
169return speechTranscription.alternatives
170
171.map((alternative) => {
172
173// Get the word segments from the speech transcription
174
175const words = alternative.words ?? [];
176
177
178
179// Place the word segments into groups of ten
180
181const groupOfTens = words.reduce((group, word, arr) => {
182
183return (
184
185(arr % 10
186
187? group[group.length - 1].push(word)
188
189: group.push([word])) && group
190
191);
192
193}, []);
194
195
196
197// Map through the word groups and build a sentence with the start time and end time
198
199return groupOfTens.map((group) => {
200
201// Start offset time in seconds
202
203const startOffset =
204
205parseInt(group[0].startTime.seconds ?? 0) +
206
207(group[0].startTime.nanos ?? 0) / 1000000000;
208
209
210
211// End offset time in seconds
212
213const endOffset =
214
215parseInt(group[group.length - 1].endTime.seconds ?? 0) +
216
217(group[group.length - 1].endTime.nanos ?? 0) / 1000000000;
218
219
220
221return {
222
223startTime: startOffset,
224
225endTime: endOffset,
226
227sentence: group.map((word) => word.word).join(" "),
228
229};
230
231});
232
233})
234
235.flat();
236
237})
238
239.flat();
240
241
242
243// Build the subtitle file content
244
245const subtitleContent = allSentences
246
247.map((sentence, index) => {
248
249// Format the start time
250
251const startTime = intervalToDuration({
252
253start: 0,
254
255end: sentence.startTime * 1000,
256
257});
258
259
260
261// Format the end time
262
263const endTime = intervalToDuration({
264
265start: 0,
266
267end: sentence.endTime * 1000,
268
269});
270
271
272
273return `${index + 1}\n${startTime.hours}:${startTime.minutes}:${
274
275startTime.seconds
276
277},000 --> ${endTime.hours}:${endTime.minutes}:${endTime.seconds},000\n${
278
279sentence.sentence
280
281}`;
282
283})
284
285.join("\n\n");
286
287
288
289const subtitlePath = `public/subtitles/subtitle.srt`;
290
291
292
293// Write the subtitle file to the filesystem
294
295await fs.writeFile(subtitlePath, subtitleContent);
296
297
298
299// Upload the subtitle file to Cloudinary
300
301const subtitleUploadResult = await handleCloudinaryUpload({
302
303path: subtitlePath,
304
305folder: false,
306
307});
308
309
310
311// Delete the subtitle file from the filesystem
312
313await fs.unlink(subtitlePath);
314
315
316
317// Upload the video file to Cloudinary and apply the subtitle file as an overlay/layer
318
319const videoUploadResult = await handleCloudinaryUpload({
320
321path: video.filepath,
322
323folder: true,
324
325transformation: [
326
327{
328
329background: "black",
330
331color: "yellow",
332
333overlay: {
334
335font_family: "Arial",
336
337font_size: "32",
338
339font_weight: "bold",
340
341resource_type: "subtitles",
342
343public_id: subtitleUploadResult.public_id,
344
345},
346
347},
348
349{ flags: "layer_apply" },
350
351],
352
353});
354
355
356
357return videoUploadResult;
358
359};

This is where the magic happens. At the top, we export a custom config object. This config object tells Next.js not to use the default body parser since we'll be parsing the form data on our own. Read about custom config in API routes here. The default exported function named handler is standard for Next.js API routes. We use a switch statement to only handle GET and POST requests.

handleGetRequest gets all the uploaded resources by calling the handleGetCloudinaryUploads function that we created earlier.

handlePostRequest takes in the incoming request object. We use the parseForm method that we created in the parse-form.js file to get the form data. We then get the video file, get the base64 string and pass it to analyzeVideoTranscript. This video transcribes the video using Google Video Intelligence. The following is the structure of the data that we get back

1{
2
3segment: {
4
5startTimeOffset: {
6
7seconds: string;
8
9nanos: number;
10
11};
12
13endTimeOffset: {
14
15seconds: string;
16
17nanos: number;
18
19};
20
21};
22
23speechTranscriptions: [
24
25{
26
27alternatives: [
28
29{
30
31transcript: string;
32
33confidence: number;
34
35words: [
36
37{
38
39startTime: {
40
41seconds: string;
42
43nanos: number;
44
45};
46
47endTime: {
48
49seconds: string;
50
51nanos: number;
52
53};
54
55word: string;
56
57}
58
59];
60
61}
62
63];
64
65languageCode: string;
66
67}
68
69];
70
71}

You can also check out some sample data here

We need to convert that to the following structure

1[
2
3{
4
5startTime: number;
6
7endTime: number;
8
9sentence: string;
10
11}
12
13]

To achieve this we map through the annotations.speechTranscriptions, then the alternatives for each speech transcription. Google returns each word separate from its start and end time. We put those words in groups of ten so that we can form sentences with ten words. We don't want our sentences to be too long. We then join the group of words to make a sentence and flatten everything.

Next, we need to create a subtitle file. Let's have a look at the structure of a subtitles(srt) file.

1number
2
3hour:minute:second,millisecond --> hour:minute:second,millisecond
4
5sentence
6
7
8
9number
10
11hour:minute:second,millisecond --> hour:minute:second,millisecond
12
13sentence

For example

11
2
300:01:20,000 --> 00:01:30,000
4
5This is the first frame
6
7
8
92
10
1100:01:31,000 --> 00:01:40,000
12
13This is the second frame

We model our data into this format in the following piece of code

1// Build the subtitle file content
2
3const subtitleContent = allSentences
4
5.map((sentence, index) => {
6
7// Format the start time
8
9const startTime = intervalToDuration({
10
11start: 0,
12
13end: sentence.startTime * 1000,
14
15});
16
17
18
19// Format the end time
20
21const endTime = intervalToDuration({
22
23start: 0,
24
25end: sentence.endTime * 1000,
26
27});
28
29
30
31return `${index + 1}\n${startTime.hours}:${startTime.minutes}:${
32
33startTime.seconds
34
35},000 --> ${endTime.hours}:${endTime.minutes}:${endTime.seconds},000\n${
36
37sentence.sentence
38
39}`;
40
41})
42
43.join("\n\n");

We then use writeFile to create a new subtitle file. We upload the subtitle file to cloudinary. After this is done we upload our video to cloudinary and apply the subtitle file as a layer. Read about how this works in the cloudinary docs.

Moving on to the [...id].js file. Paste the following inside pages/api/videos/[...id].js

1// pages/api/videos/[...id].js`
2
3
4
5import { NextApiRequest, NextApiResponse } from "next";
6
7import {
8
9handleCloudinaryDelete,
10
11handleGetCloudinaryUpload,
12
13} from "../../../lib/cloudinary";
14
15
16
17/**
18
19*
20
21* @param {NextApiRequest} req
22
23* @param {NextApiResponse} res
24
25*/
26
27export default async function handler(req, res) {
28
29const id = Array.isArray(req.query.id)
30
31? req.query.id.join("/")
32
33: req.query.id;
34
35
36
37switch (req.method) {
38
39case "GET": {
40
41try {
42
43const result = await handleGetRequest(id);
44
45
46
47return res.status(200).json({ message: "Success", result });
48
49} catch (error) {
50
51console.error(error);
52
53return res.status(400).json({ message: "Error", error });
54
55}
56
57}
58
59
60
61case "DELETE": {
62
63try {
64
65const result = await handleDeleteRequest(id);
66
67
68
69return res.status(200).json({ message: "Success", result });
70
71} catch (error) {
72
73console.error(error);
74
75return res.status(400).json({ message: "Error", error });
76
77}
78
79}
80
81
82
83default: {
84
85return res.status(405).json({ message: "Method Not Allowed" });
86
87}
88
89}
90
91}
92
93
94
95/**
96
97* Gets a single resource from Cloudinary.
98
99*
100
101* @param {string} id Public ID of the video to get
102
103*/
104
105const handleGetRequest = async (id) => {
106
107const upload = await handleGetCloudinaryUpload(id);
108
109
110
111return upload;
112
113};
114
115
116
117/**
118
119* Handles the DELETE request to the API route.
120
121*
122
123* @param {string} id Public ID of the video to delete
124
125*/
126
127const handleDeleteRequest = (id) => {
128
129// Delete the uploaded image from Cloudinary
130
131return handleCloudinaryDelete([id]);
132
133};

handleGetRequest calls the handleGetCloudinaryUpload with the public id of the video and gets the uploaded video.

handleDeleteRequest just deletes the resource with the given public id.

Let's move on to the frontend. Add the following code to styles/globals.css

1:root {
2
3--color-primary: #ff0000;
4
5--color-primary-light: #ff4444;
6
7}
8
9
10
11.btn {
12
13display: inline-block;
14
15padding: 0.5rem 1rem;
16
17background: var(--color-primary);
18
19color: #ffffff;
20
21border: none;
22
23border-radius: 0.25rem;
24
25cursor: pointer;
26
27font-size: 1rem;
28
29font-weight: bold;
30
31text-transform: uppercase;
32
33text-decoration: none;
34
35text-align: center;
36
37transition: all 0.2s ease-in-out;
38
39}
40
41
42
43.btn:hover {
44
45background: var(--color-primary-light);
46
47box-shadow: 0 0 0.25rem 0 rgba(0, 0, 0, 0.25);
48
49}

These are just a few styles to help us with the UI.

Create a new folder at the root of your project and name it components. This folder will hold our shared components. Create a new file under components called Layout.js and paste the following code inside.

1// components/Layout.js
2
3
4import Head from "next/head";
5
6import Link from "next/link";
7
8
9export default function Layout({ children }) {
10
11return (
12
13<div>
14
15<Head>
16
17<title>
18
19Add subtitles to videos using google video intelligence and cloudinary
20
21</title>
22
23<meta
24
25name="description"
26
27content=" Add subtitles to videos using google video intelligence and cloudinary"
28
29/>
30
31<link rel="icon" href="/favicon.ico" />
32
33</Head>
34
35<nav>
36
37<ul>
38
39<li>
40
41<Link href="/">
42
43<a className="btn">Home</a>
44
45</Link>
46
47</li>
48
49<li>
50
51<Link href="/videos">
52
53<a className="btn">Videos</a>
54
55</Link>
56
57</li>
58
59</ul>
60
61</nav>
62
63<main>{children}</main>
64
65<style jsx>{`
66
67nav {
68
69background-color: #f0f0f0;
70
71min-height: 100px;
72
73display: flex;
74
75align-items: center;
76
77}
78
79
80
81nav ul {
82
83list-style: none;
84
85padding: 0 32px;
86
87flex: 1;
88
89display: flex;
90
91flex-flow: row nowrap;
92
93justify-content: center;
94
95gap: 8px;
96
97}
98
99`}</style>
100
101</div>
102
103);
104
105}

We'll be wrapping our pages in this component. It allows us to have a consistent layout. Paste the following code inside pages/index.js

1// pages/index.js
2
3
4
5import { useRouter } from "next/router";
6
7import { useState } from "react";
8
9import Layout from "../components/Layout";
10
11
12
13export default function Home() {
14
15const router = useRouter();
16
17const [isLoading, setIsLoading] = useState(false);
18
19
20
21const handleFormSubmit = async (event) => {
22
23event.preventDefault();
24
25
26
27try {
28
29setIsLoading(true);
30
31
32
33const formData = new FormData(event.target);
34
35
36
37const response = await fetch("/api/videos", {
38
39method: "POST",
40
41body: formData,
42
43});
44
45
46
47const data = await response.json();
48
49
50
51if (!response.ok) {
52
53throw data;
54
55}
56
57
58
59router.push("/videos");
60
61} catch (error) {
62
63console.error(error);
64
65} finally {
66
67setIsLoading(false);
68
69}
70
71};
72
73
74
75return (
76
77<Layout>
78
79<div className="wrapper">
80
81<form onSubmit={handleFormSubmit}>
82
83<h2>Upload video file</h2>
84
85<div className="input-group">
86
87<label htmlFor="video">Video File</label>
88
89<input
90
91type="file"
92
93name="video"
94
95id="video"
96
97accept=".mp4,.mov,.mpeg4,.avi"
98
99multiple={false}
100
101required
102
103disabled={isLoading}
104
105/>
106
107</div>
108
109<button className="btn" type="submit" disabled={isLoading}>
110
111Upload
112
113</button>
114
115<button className="btn" type="reset" disabled={isLoading}>
116
117Cancel
118
119</button>
120
121</form>
122
123</div>
124
125<style jsx>{`
126
127div.wrapper {
128
129}
130
131
132
133div.wrapper > form {
134
135margin: 64px auto;
136
137background-color: #fdd8d8;
138
139padding: 40px 20px;
140
141width: 60%;
142
143display: flex;
144
145flex-flow: column;
146
147gap: 8px;
148
149border-radius: 0.25rem;
150
151}
152
153
154
155div.wrapper > form > div.input-group {
156
157display: flex;
158
159flex-flow: column;
160
161gap: 8px;
162
163}
164
165
166
167div.wrapper > form > div.input-group > label {
168
169font-weight: bold;
170
171}
172
173
174
175div.wrapper > form > div.input-group > input {
176
177background-color: #f5f5f5;
178
179}
180
181
182
183div.wrapper > form > button {
184
185height: 50px;
186
187}
188
189`}</style>
190
191</Layout>
192
193);
194
195}

This is a simple page with a form for uploading the video that we want to add subtitles to. handleFormSubmit makes a POST request to /api/videos with the form data and then navigates to the /videos page upon success.

Create a new folder under the pages folder and call it videos. Create two files under pages/videos called index.js and [...id].js. Please note that this is not the same as the pages/api/videos folder. Paste the following code inside pages/videos/index.js

1// pages/videos/index.js
2
3
4
5import Link from "next/link";
6
7import Image from "next/image";
8
9import { useCallback, useEffect, useState } from "react";
10
11import Layout from "../../components/Layout";
12
13
14
15export default function VideosPage() {
16
17const [isLoading, setIsLoading] = useState(false);
18
19const [videos, setVideos] = useState([]);
20
21
22
23const getVideos = useCallback(async () => {
24
25try {
26
27setIsLoading(true);
28
29
30
31const response = await fetch("/api/videos", {
32
33method: "GET",
34
35});
36
37
38
39const data = await response.json();
40
41
42
43if (!response.ok) {
44
45throw data;
46
47}
48
49
50
51setVideos(data.result.resources);
52
53console.log(data);
54
55} catch (error) {
56
57// TODO: Show error message to the user
58
59console.error(error);
60
61} finally {
62
63setIsLoading(false);
64
65}
66
67}, []);
68
69
70
71useEffect(() => {
72
73getVideos();
74
75}, [getVideos]);
76
77
78
79return (
80
81<Layout>
82
83<div className="wrapper">
84
85<div className="videos-wrapper">
86
87{videos.map((video, index) => {
88
89const splitVideoUrl = video.secure_url.split(".");
90
91
92
93splitVideoUrl[splitVideoUrl.length - 1] = "jpg";
94
95
96
97const thumbnail = splitVideoUrl.join(".");
98
99
100
101return (
102
103<div className="video-wrapper" key={`video-${index}`}>
104
105<div className="thumbnail">
106
107<Image
108
109src={thumbnail}
110
111alt={video.secure_url}
112
113layout="fill"
114
115></Image>
116
117</div>
118
119<div className="actions">
120
121<Link
122
123href="/videos/[...id]"
124
125as={`/videos/${video.public_id}`}
126
127>
128
129<a>Open Video</a>
130
131</Link>
132
133</div>
134
135</div>
136
137);
138
139})}
140
141</div>
142
143</div>
144
145
146
147{!isLoading && videos.length === 0 ? (
148
149<div className="no-videos">
150
151<b>No videos yet</b>
152
153<Link href="/" passHref>
154
155<button className="btn">Upload Video</button>
156
157</Link>
158
159</div>
160
161) : null}
162
163
164
165{isLoading ? (
166
167<div className="loading">
168
169<b>Loading...</b>
170
171</div>
172
173) : null}
174
175
176
177<style jsx>{`
178
179div.wrapper {
180
181min-height: 100vh;
182
183}
184
185
186
187div.wrapper h1 {
188
189text-align: center;
190
191}
192
193
194
195div.wrapper div.videos-wrapper {
196
197padding: 20px;
198
199display: flex;
200
201flex-flow: row wrap;
202
203gap: 20px;
204
205}
206
207
208
209div.wrapper div.videos-wrapper div.video-wrapper {
210
211flex: 0 0 400px;
212
213height: 400px;
214
215}
216
217
218
219div.wrapper div.videos-wrapper div.video-wrapper div.thumbnail {
220
221position: relative;
222
223width: 100%;
224
225height: 80%;
226
227}
228
229
230
231div.loading,
232
233div.no-videos {
234
235height: 100vh;
236
237display: flex;
238
239flex-flow: column;
240
241justify-content: center;
242
243align-items: center;
244
245gap: 8px;
246
247}
248
249`}</style>
250
251</Layout>
252
253);
254
255}

This page will call getVideos when it renders. getVideos makes a GET request to /api/videos to get all the uploaded videos. You can read about the useCallback and useEffect react hooks from the react docs. We then show thumbnails of the videos. See here on how to generate a thumbnail of a cloudinary video.

And now for the final page. Paste the following inside pages/videos/[...id].js

1// pages/videos/[...id].js
2
3
4
5import { useRouter } from "next/router";
6
7import { useCallback, useEffect, useState } from "react";
8
9import Layout from "../../components/Layout";
10
11
12
13export default function VideoPage() {
14
15const router = useRouter();
16
17
18
19const id = Array.isArray(router.query.id)
20
21? router.query.id.join("/")
22
23: router.query.id;
24
25
26
27const [isLoading, setIsLoading] = useState(false);
28
29const [video, setVideo] = useState(null);
30
31
32
33const getVideo = useCallback(async () => {
34
35try {
36
37setIsLoading(true);
38
39const response = await fetch(`/api/videos/${id}`, {
40
41method: "GET",
42
43});
44
45
46
47const data = await response.json();
48
49
50
51if (!response.ok) {
52
53throw data;
54
55}
56
57
58
59setVideo(data.result);
60
61console.log(data);
62
63} catch (error) {
64
65// TODO: Show error message to the user
66
67console.error(error);
68
69} finally {
70
71setIsLoading(false);
72
73}
74
75}, [id]);
76
77
78
79useEffect(() => {
80
81getVideo();
82
83}, [getVideo]);
84
85
86
87const handleDownload = async () => {
88
89try {
90
91setIsLoading(true);
92
93
94
95const response = await fetch(video.secure_url, {});
96
97
98
99if (response.ok) {
100
101const blob = await response.blob();
102
103
104
105const fileUrl = URL.createObjectURL(blob);
106
107
108
109const a = document.createElement("a");
110
111a.href = fileUrl;
112
113a.download = `${video.public_id.replace("/", "-")}.${video.format}`;
114
115document.body.appendChild(a);
116
117a.click();
118
119a.remove();
120
121return;
122
123}
124
125
126
127throw await response.json();
128
129} catch (error) {
130
131// TODO: Show error message to the user
132
133console.error(error);
134
135} finally {
136
137setIsLoading(false);
138
139}
140
141};
142
143
144
145const handleDelete = async () => {
146
147try {
148
149setIsLoading(true);
150
151
152
153const response = await fetch(`/api/videos/${id}`, {
154
155method: "DELETE",
156
157});
158
159
160
161const data = await response.json();
162
163
164
165if (!response.ok) {
166
167throw data;
168
169}
170
171
172
173router.replace("/videos");
174
175} catch (error) {
176
177console.error(error);
178
179} finally {
180
181setIsLoading(false);
182
183}
184
185};
186
187
188
189return (
190
191<Layout>
192
193{video && !isLoading ? (
194
195<div className="wrapper">
196
197<div className="video-wrapper">
198
199<video src={video.secure_url} controls></video>
200
201<div className="actions">
202
203<button
204
205className="btn"
206
207onClick={handleDownload}
208
209disabled={isLoading}
210
211>
212
213Download
214
215</button>
216
217<button
218
219className="btn"
220
221onClick={handleDelete}
222
223disabled={isLoading}
224
225>
226
227Delete
228
229</button>
230
231</div>
232
233</div>
234
235</div>
236
237) : null}
238
239
240
241{isLoading ? (
242
243<div className="loading">
244
245<b>Loading...</b>
246
247</div>
248
249) : null}
250
251
252
253<style jsx>{`
254
255div.wrapper {
256
257}
258
259
260
261div.wrapper > div.video-wrapper {
262
263width: 80%;
264
265margin: 20px auto;
266
267display: flex;
268
269flex-flow: column;
270
271gap: 8px;
272
273}
274
275
276
277div.wrapper > div.video-wrapper > video {
278
279width: 100%;
280
281}
282
283
284
285div.wrapper > div.video-wrapper > div.actions {
286
287display: flex;
288
289flex-flow: row;
290
291gap: 8px;
292
293}
294
295
296
297div.loading {
298
299height: 100vh;
300
301display: flex;
302
303justify-content: center;
304
305align-items: center;
306
307}
308
309`}</style>
310
311</Layout>
312
313);
314
315}

getVideo makes a GET request to /api/videos/:id to get the video with the given id. handleDownload just downloads the video file. handleDelete makes a DELETE request to /api/videos/:id to delete the video with the given id.

For the final piece of the puzzle. Add the following to next.config.js.

1module.exports = {
2
3// ... other options
4
5images: {
6
7domains: ["res.cloudinary.com"],
8
9},
10
11};

This is because we're using the Image component from Next.js. We need to add the cloudinary domain so that images from that domain can be optimized. Read more about this here

You can now run your application by running the following command

1npm run dev

And that's a wrap for this tutorial. You can find the full code on my Github. Please note that this is just a simple demonstration. There's a lot of ways you could optimize your application. Have a look at Google's long running operations and Cloudinary's notifications

Eugene Musebe

Software Developer

I’m a full-stack software developer, content creator, and tech community builder based in Nairobi, Kenya. I am addicted to learning new technologies and loves working with like-minded people.